Phantom degrees of freedom in joint estimation and control

ABSTRACT

Methods, apparatus, and systems for operating a surgical system. In accordance with a method, a position of a surgical instrument is measured, the surgical instrument being included in a mechanical assembly having a plurality of joints and a first number of degrees of freedom, the position of the surgical instrument being measured for each of a second number of degrees of freedom of the surgical instrument. The method further includes estimating a position of each of the joints, where estimating the position of each joint includes applying the position measurements to at least one kinematic model of the mechanical assembly, the kinematic model having a third number of degrees of freedom greater than the first number of degrees of freedom. The method further includes controlling the mechanical assembly based on the estimated position of the joints.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.61/683,607, filed Aug. 15, 2012, which is incorporated by referenceherein in its entirety for all purposes.

The present application is generally related to the followingcommonly-owned applications: U.S. Provisional Application No. 61/654,764filed Jun. 1, 2012, entitled “Commanded Reconfiguration of a SurgicalManipulator Using the Null Space”, U.S. application Ser. No. 12/494,695filed Jun. 30, 2009, entitled “Control of Medical Robotic SystemManipulator About Kinematic Singularities;” U.S. application Ser. No.12/406,004 filed Mar. 17, 2009, entitled “Master Controller HavingRedundant Degrees of Freedom and Added Forces to Create InternalMotion;” U.S. application Ser. No. 11/133,423 filed May 19, 2005 (U.S.Pat. No. 8,004,229), entitled “Software Center and Highly ConfigurableRobotic Systems for Surgery and Other Uses;” U.S. application Ser. No.10/957,077 filed Sep. 30, 2004 (U.S. Pat. No. 7,594,912), entitled“Offset Remote Center Manipulator For Robotic Surgery;” and U.S.application Ser. No. 09/398,507 filed Sep. 17, 1999 (U.S. Pat. No.6,714,839), entitled “Master Having Redundant Degrees of Freedom,” thedisclosures of which are incorporated herein by reference in theirentireties.

BACKGROUND

Embodiments of the present invention generally provide improved surgicaland/or robotic devices, systems, and methods.

Minimally invasive medical techniques are aimed at reducing the amountof extraneous tissue which is damaged during diagnostic or surgicalprocedures, thereby reducing patient recovery time, discomfort, anddeleterious side effects. Millions of surgeries are performed each yearin the United States. Many of these surgeries can potentially beperformed in a minimally invasive manner. However, only a relativelysmall number of surgeries currently use these techniques due tolimitations in minimally invasive surgical instruments and techniquesand the additional surgical training required to master them.

Minimally invasive telesurgical systems for use in surgery are beingdeveloped to increase a surgeon's dexterity as well as to allow asurgeon to operate on a patient from a remote location. Telesurgery is ageneral term for surgical systems where the surgeon uses some form ofremote control, e.g., a servomechanism, or the like, to manipulatesurgical instrument movements rather than directly holding and movingthe instruments by hand. In such a telesurgery system, the surgeon isprovided with an image of the surgical site at the remote location.While viewing typically a three-dimensional image of the surgical siteon a suitable viewer or display, the surgeon performs the surgicalprocedures on the patient by manipulating master control input devices,which in turn control the motion of robotic instruments. The roboticsurgical instruments can be inserted through small, minimally invasivesurgical apertures to treat tissues at surgical sites within thepatient, such apertures resulting in the trauma typically associatedwith open surgery. These robotic systems can move the working ends ofthe surgical instruments with sufficient dexterity to perform quiteintricate surgical tasks, often by pivoting shafts of the instruments atthe minimally invasive aperture, sliding of the shaft axially throughthe aperture, rotating of the shaft within the aperture, and/or thelike.

The servomechanism used for telesurgery will often accept input from twomaster controllers (one for each of the surgeon's hands) and may includetwo or more robotic arms or manipulators. Mapping of the hand movementsto the image of the robotic instruments displayed by the image capturedevice can help provide the surgeon with accurate control over theinstruments associated with each hand. In many surgical robotic systems,one or more additional robotic manipulator arms are included for movingan endoscope or other image capture device, additional surgicalinstruments, or the like.

A variety of structural arrangements can be used to support the surgicalinstrument at the surgical site during robotic surgery. The drivenlinkage or “slave” is often called a robotic surgical manipulator, andexemplary linkage arrangements for use as a robotic surgical manipulatorduring minimally invasive robotic surgery are described in U.S.Provisional Application No. 61/654,764 filed Jun. 1, 2012, entitled“Commanded Reconfiguration of a Surgical Manipulator Using the NullSpace”, and U.S. Pat. Nos. 6,758,843; 6,246,200; and 5,800,423, the fulldisclosures of which are incorporated herein by reference in theirentirety. These linkages often make use of a parallelogram arrangementto hold an instrument having a shaft. Such a manipulator structure canconstrain movement of the instrument so that the instrument shaft pivotsabout a remote center of spherical rotation positioned in space alongthe length of the rigid shaft. By aligning this center of rotation withthe incision point to the internal surgical site (for example, with atrocar or cannula at an abdominal wall during laparoscopic surgery), anend effector of the surgical instrument can be positioned safely bymoving the proximal end of the shaft using the manipulator linkagewithout imposing dangerous forces against the abdominal wall.Alternative manipulator structures are described, for example, in U.S.Pat. Nos. 7,594,912, 6,702,805; 6,676,669; 5,855,583; 5,808,665;5,445,166; and 5,184,601, the full disclosures of which are incorporatedherein by reference in their entirety.

While the new robotic surgical systems and devices have proven highlyeffective and advantageous, still further improvements would bedesirable. In some cases, the master controller(s) used by the surgeonhave a number of degrees of freedom more than or equal to the number ofdegrees of freedom which the end effectors of the remotely controlledrobotic manipulator arms and/or tools have. In such cases, controllersthat are used to control the robotic manipulator arms and/or tools maybecome overconstrained. For example, where the remote tool is a rigidendoscope extending through a minimally invasive aperture, twoorientational degrees of freedom may not be available within theworkspace (those associated with a tool wrist near an end effector,e.g., wrist pitch and yaw). Accordingly, the robotic manipulator withendoscope only has four degrees of freedom at its tip. In practice,these mathematical problems can become tangible, resulting in asluggish, unresponsive feel to the surgeon which is undesirable. Furtherproblems can arise when tools having different degrees of freedom areused with the same robotic surgical manipulator. For example, a surgeonmay wish to use jaws having three degrees of freedom, and then replacethe jaws with a suction device having two degree of freedom. Evenfurther problems can arise when using estimated joint positions tocontrol tool movements in situations where input and output degrees offreedom differ. Such situations may result in numerical errors beingimposed into the joint position estimations resulting in undesired toolmovements.

For these and other reasons, it would be advantageous to provideimproved devices, systems, and methods for surgery, robotic surgery, andother robotic applications. It would be particularly beneficial if theseimproved technologies provided the ability to effectively controlrobotic manipulator arms and/or tools with end effectors having a numberof degrees of freedom fewer than the number of degrees of freedom of amaster controller manipulated by a surgeon. It would be even morebeneficial if these improved technologies allowed the same computationengine to be used for all instruments of the robotic system, therebyreducing controller complexity and costs while increasing flexibility.

BRIEF SUMMARY

Embodiments of the present invention generally provide improved roboticand/or surgical devices, systems, and methods. In one embodiment, amethod of operating a surgical system is disclosed. The method includesvarious operations, including measuring a position of a surgicalinstrument. The surgical instrument is included in a mechanical assemblyhaving a plurality of joints, and the position of the surgicalinstrument is measured for each of a first number of degrees of freedomof the surgical instrument. The method further includes estimating aposition of each of the joints, where estimating the position of eachjoint includes applying the position measurements to at least onekinematic model of the mechanical assembly. The kinematic model has asecond number of degrees of freedom greater than the first number ofdegrees of freedom. The method further includes controlling themechanical assembly based on the estimated position of the joints.

In accordance with another embodiment, a surgical system for performingminimally invasive surgery through an aperture of a patient isdisclosed. The system includes a robotic manipulator assembly includinga surgical tool. The robotic manipulator assembly has a plurality ofjoints, and is operable to position the surgical tool at the aperture ofthe patient. The system further includes a tool position measuringdevice operable to measure a position of the surgical tool for each of afirst number of degrees of freedom of the surgical tool. The system alsoincludes a controller. The controller is operable to perform a varietyof functions. For example, the controller is operable to estimate aposition of each of the joints, where estimating the position of eachjoint includes applying the position measurements to a kinematic modelof the robotic manipulator assembly. The kinematic model has a secondnumber of degrees of freedom greater than the first number of degrees offreedom. The controller is further operable to control the roboticmanipulator assembly based on the estimated position of the joints.

In accordance with yet another embodiment, method for controlling themovement of a mechanical body is disclosed. The method includesreceiving control information for controlling the position of amechanical body. The mechanical body has a first number of degrees offreedom. The method also includes generating a plurality of individualcontrol outputs by applying the received control information to akinematic model. The kinematic model has a second number of degrees offreedom greater than the first number of degrees of freedom. Each of theindividual control outputs are configured to affect control of acorresponding one of the second number of degrees of freedom. The methodfurther includes transmitting a subset of the plurality of individualcontrol outputs for use in controlling the first number of degrees offreedom of the mechanical body.

For a fuller understanding of the nature and advantages of the presentinvention, reference should be made to the ensuing detailed descriptionand accompanying drawings. Other aspects, objects and advantages of theinvention will be apparent from the drawings and detailed descriptionthat follows.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is an overhead view of a robotic surgical system in accordancewith embodiments of the present invention.

FIG. 1B diagrammatically illustrates the robotic surgical system of FIG.1A.

FIG. 2 is a perspective view of the surgeon console of FIG. 1A.

FIG. 3 is a perspective view of the electronics cart of FIG. 1A.

FIG. 4 is a perspective view of a patient side cart having a pluralityof manipulator arms each supporting a surgical instrument.

FIG. 5 is a perspective view of a manipulator arm in accordance with anembodiment.

FIG. 6A is a perspective view of a robotic surgery tool that includes anend effector having opposing clamping jaws in accordance with anembodiment.

FIG. 6B illustrates a wristed endoscope in accordance with anembodiment.

FIG. 6C is a perspective view of the distal end of an overtube withsuction ports in accordance with an embodiment.

FIG. 6D illustrates a non-wristed endoscope in accordance with anembodiment.

FIG. 7A is a perspective view of a master control input device inaccordance with an embodiment.

FIG. 7B is a perspective view of a gimbal or wrist of the input deviceof FIG. 7A.

FIG. 7C is a perspective view of an articulated arm of the input deviceof FIG. 7A.

FIG. 8 is a block diagram showing a simplified system for controlling amechanical body having fewer degrees of freedom than thosemathematically modeled

FIG. 9 is a block diagram of an actuator according an embodiment.

FIG. 10A is a block diagram showing a simplified system for controllinga manipulator assembly using an input device in accordance with a firstembodiment.

FIG. 10B is a block diagram showing a simplified system for controllinga manipulator assembly using an input device in accordance with a secondembodiment.

FIG. 10C is a block diagram showing a simplified system for controllinga manipulator assembly using an input device in accordance with a thirdembodiment.

FIG. 10D is a block diagram showing a simplified system for controllinga manipulator assembly using an input device in accordance with a fourthembodiment.

FIG. 11A is a block diagram showing a simplified system for controllinga manipulator assembly using a tool position measuring device inaccordance with a first embodiment.

FIG. 11B is a block diagram showing a simplified system for controllinga manipulator assembly using a tool position measuring device inaccordance with a second embodiment.

FIG. 12A is a manipulator assembly having more than five degrees offreedom according to an embodiment.

FIG. 12B depicts a block diagram illustrating the calculation of jointpositions of multiple manipulator assembly parts according to anembodiment.

FIG. 13 is a flowchart showing a process for controlling manipulatorarms, tools, and/or end effectors using an input device according to afirst embodiment.

FIG. 14 is a flowchart showing a process for controlling manipulatorarms, tools, and/or end effectors using an input device according to asecond embodiment.

DETAILED DESCRIPTION

Embodiments of the present invention generally provide improvedtechniques for controlling the movement of mechanical bodies. Someembodiments are particularly advantageous for use with surgical roboticsystems in which a plurality of surgical tools or instruments aremounted on and moved by an associated plurality of robotic manipulatorsduring a surgical procedure. The robotic systems will often comprisetelerobotic, telesurgical, and/or telepresence systems that includeprocessors configured as master-slave controllers. By providing roboticsystems employing processors appropriately configured to movemanipulator assemblies with articulated linkages having relatively largenumbers of degrees of freedom, the motion of the linkages can betailored for work through a minimally invasive access site.

The robotic manipulator assemblies described herein will often include arobotic manipulator and a tool mounted thereon (the tool oftencomprising a surgical instrument in surgical versions), although theterm “robotic assembly” will also encompass the manipulator without thetool mounted thereon. The term “tool” encompasses both general orindustrial robotic tools and specialized robotic surgical instruments,with these later structures often including an end effector which issuitable for manipulation of tissue, treatment of tissue, imaging oftissue, or the like. The tool/manipulator interface will often be aquick disconnect tool holder or coupling, allowing rapid removal andreplacement of the tool with an alternate tool. The manipulator assemblywill often have a base which is fixed in space during at least a portionof a robotic procedure, and the manipulator assembly may include anumber of degrees of freedom between the base and an end effector of thetool. For example, the manipulator assembly may include kinematicdegrees of freedom of a manipulator as well as kinematic degrees offreedom of a tool connected to the manipulator. The combination of thesemay be referred to herein as “manipulator degrees of freedom”, and aretypically defined in joint space (described below). Actuation of the endeffector (such as opening or closing of the jaws of a gripping device,energizing an electrosurgical paddle, activating air pressure for avacuum, or the like) will often be separate from, and in addition to,these manipulator assembly degrees of freedom. These may be referred toherein as “actuation degrees of freedom”.

The end effector (or, more generally, the control frame, as describedbelow) will typically move in the workspace with between two and sixdegrees of freedom. The degrees of freedom of the end effector (or, moregenerally, the degrees of freedom of the control frame) may be referredto herein as “end effector degrees of freedom”, and are typicallydefined in Cartesian space (described below). As used herein, the term“position” encompasses both location and orientation. Hence, a change ina position of an end effector (for example) may involve a translation ofthe end effector from a first location to a second location, a rotationof the end effector from a first orientation to a second orientation, ora combination of both. When used for minimally invasive robotic surgery,movement of the manipulator assembly may be controlled by a processor ofthe system so that a shaft or intermediate portion of the tool orinstrument is constrained to a safe motion through a minimally invasivesurgical access site or other aperture. Such motion may include, forexample, axial insertion of the shaft through the aperture site into asurgical workspace, rotation of the shaft about its axis, and pivotalmotion of the shaft about a pivot point at the aperture site.

Many of the manipulator assemblies described herein have fewer degreesof freedom available for use than those that are typically associatedwith full control over the positioning of an end effector in a workspace(where full control of the end effector requires end effector degrees offreedom including three independent translations and three independentorientations). That is, the manipulator assemblies may have aninsufficient number or type of degrees of freedom for independentlycontrolling the six end effector degrees of freedom. For example, arigid endoscope tip without an articulating wrist may be missing one ortwo degrees of freedom at the wrist, such as wrist pitch and/or yaw.Accordingly, the endoscope may have only four or five degrees of freedomfor positioning the end effector, rather than six, thus potentiallyconstraining the motion of the endoscope.

However, some of the manipulator assemblies described herein have agreater number of degrees of freedom than that required to fully controlthe positioning of the end effector (where full control of the endeffector requires end effector degrees of freedom including threeindependent translations and three independent orientations), but due tothe type or arrangement of the joints of the manipulator assemblies, themanipulator assemblies still cannot fully control the positioning of theend effector. For example, a manipulator assembly may have sevenmanipulator degrees of freedom, but three of those are redundant. As aresult, the end effector effectively has five degrees of freedom.

Regardless of the number of degrees of freedom available for controllingthe position of the end effector, the manipulator assemblies describedherein may also facilitate additional degrees of freedom for actuating atool (i.e., actuation degrees of freedom). For example, the manipulatorassemblies may be configured to mount a tool having an electrocauteryprobe operable to, e.g., heat select tissue upon activation. For anotherexample, the manipulator assemblies may be configured to mount a toolhaving a vacuum operable to, e.g., apply suction forces around selecttissue upon activation. In such cases, these additional degrees offreedom are not kinematic, and therefore do not affect the position(i.e., location and orientation) of the end effector.

The term “state” of a joint or the like will often herein refer to thecontrol variables associated with the joint. For example, the state ofan angular joint can refer to the angle defined by that joint within itsrange of motion, and/or to the angular velocity of the joint. Similarly,the state of an axial or prismatic joint may refer to the joint's axialposition, and/or to its axial velocity. While many of the controllersdescribed herein comprise velocity controllers, they often also havesome position control aspects. Alternative embodiments may relyprimarily or entirely on position controllers, acceleration controllers,or the like. Many aspects of control systems that can be used in suchdevices are more fully described in U.S. Pat. No. 6,699,177, the fulldisclosure of which is incorporated herein by reference. Hence, so longas the movements described are based on the associated calculations, thecalculations of movements of the joints and movements of an end effectordescribed herein may be performed using a position control algorithm, avelocity control algorithm, a combination of both, and/or the like.

In many embodiments, the tool of an exemplary manipulator arm pivotsabout a pivot point adjacent a minimally invasive aperture. In someembodiments, the system may utilize a hardware remote center, such asthe remote center kinematics described in U.S. Pat. No. 6,786,896, theentire contents of which are incorporated herein in its entirety. Suchsystems may utilize a double parallelogram linkage which constrains themovement of the linkages such that the shaft of the instrument supportedby the manipulator pivots about a remote center point. Alternativemechanically constrained remote center linkage systems are known and/ormay be developed in the future. In other embodiments, the system mayutilize software to achieve a remote center, such as described in U.S.Pat. No. 8,004,229, the entire contents of which are incorporated hereinby reference. In a system having a software remote center, the processorcalculates movement of the joints so as to pivot an intermediate portionof the instrument shaft about a desired pivot point, as opposed to amechanical constraint. By having the capability to compute softwarepivot points, different modes characterized by the compliance orstiffness of the system can be selectively implemented. Moreparticularly, different system modes over a range of pivotpoints/centers (e.g., moveable pivot points, passive pivot points,fixed/rigid pivot point, soft pivot points) can be implemented asdesired.

In many configurations, robotic surgical systems may include mastercontroller(s) having a number of degrees of freedom fewer than, morethan, or equal to the number of degrees of freedom which the remotelycontrolled robotic manipulator arms and/or tools have. In such cases,Jacobian based or other controllers used to control the roboticmanipulator arms and/or tools typically provide complete mathematicalsolutions and satisfactory control. For example, fully controlling theposition (i.e., location and orientation) of a rigid body can employ sixindependently controllable degrees of freedom of the rigid body, whichincludes three degrees of freedom for translations and three degrees offreedom for orientations. This lends itself nicely to a Jacobian basedcontrol algorithm in which a 6×N Jacobian matrix is used.

However, when a 6×N Jacobian controller is used to control roboticmanipulator arms and/or tools having fewer than 6 degrees of freedom,problems can be introduced since the mathematical problem isoverconstrained. For example, where the remote tool is a rigid endoscopeextending through a minimally invasive aperture (so that the endoscopepivots at the aperture), two manipulator degrees of freedom may not beavailable (those often associated with a tool wrist adjacent the endeffector, e.g., wrist pitch and yaw. Each of these two missingmanipulator degrees of freedom affects both translations andorientations of the end effector. Accordingly, the endoscope only hasfour independently controllable degrees of freedom at its tip (i.e., endeffector degrees of freedom), which can result in the aforementionedmathematical problems for a 6×N Jacobian approach. In practice, thesemathematical problems often become tangible. When, for example, theendoscope tip is commanded to either pan or tilt, since it has anon-wristed tip, it can only do one thing, and that is to do acombination of both. In other words, the endoscope tip may not beindependently controlled to pan or tilt; rather, it can only perform afixed combination of these. This can potentially result in a sluggish,unresponsive feel to the surgeon which is undesirable.

Further problems often arise when tools having different degrees offreedom are used with the same robotic surgical manipulator. Forexample, a surgeon may wish to use jaws having three kinematic degreesof freedom, and then replace the jaws with a suction device having twokinematic degrees of freedom. Since the mathematical model forcontrolling the motion of the jaws is different than that forcontrolling the motion of the suction device, the robotic system appliestwo different models to avoid the aforementioned problems resulting fromthe same mathematical model being used. For example, where themanipulator provides three degrees of freedom in addition to the degreesof freedom of the tool, controllers for controlling the motion of thejaws and the suction device may include a 6×N Jacobian based controllerand a 5×(N−1) Jacobian based controller, respectively. The use ofmultiple controllers results in an added layer of complexity that mayincrease cost and/or limit scalability for a large set of differenttools, and a 5×(N−1) Jacobian is more complicated to use due to itsreduced number of rows.

Yet further problems often arise when using estimates of the currentjoint positions of the manipulator assembly to control subsequentmovements of the joints. Joint controllers may use the combination ofdesired joint positions and estimates of the current joint positions todetermine the appropriate torque to apply to the joints so as to movethe joints closer to the desired joint positions. In situations wherethe number of degrees of freedom of the kinematic model of a manipulatorassembly are equal to the number of degrees of freedom of themanipulator assembly, but the position of a tool tip is measured in agreater number of degrees of freedom, using those measurements as inputsto the kinematic model to determine joint angles results in errors beingimposed in the resulting joint angle calculations. These errors resultin undesirable control of the manipulator assembly when those jointangle calculations are used to control motion of the manipulatorassembly.

Although manipulator assemblies having a variety of degrees of freedomare disclosed herein, including assemblies having fewer than, the samenumber as, or more than the six degrees of freedom for fully controllingthe position of an end effector, many embodiments of these assemblieslack at least one degree of freedom for fully controlling the positionof the end effector. While the manipulator assemblies may lack one ofthese degrees of freedom, the input device controlling the manipulatorassembly (e.g., a master control input device) may include the lackingdegree of freedom. In accordance with embodiments of the presentinvention, in response to an input controlling the degree(s) of freedommissing at the manipulator assembly, the other degrees of freedomavailable at the manipulator assembly may provide motions so as tosimulate control of the missing degree(s) of freedom. This may be doneby using a kinematic model of the manipulator assembly that includes andperforms calculations for the missing manipulator degree(s) of freedom.By performing such calculations, the remaining degrees of freedom of themanipulator assembly may be more effectively controlled to cause an endeffector to appear to move along the requested degree(s) of freedom.Further, the use of such a kinematic model may advantageously reduce thecomplexity of facilitating the positioning and/or actuation of toolshaving different numbers of degrees of freedom.

In the following description, various embodiments of the presentinvention will be described. For purposes of explanation, specificconfigurations and details are set forth in order to provide a thoroughunderstanding of the embodiments. However, it will also be apparent toone skilled in the art that the present invention may be practicedwithout the specific details. Furthermore, well-known features may beomitted or simplified in order not to obscure the embodiment beingdescribed.

Referring now to the drawings, in which like reference numeralsrepresent like parts throughout the several views, FIG. 1A is anoverhead view illustration of a Minimally Invasive Robotic Surgical(MIRS) system 10, in accordance with many embodiments, for use inperforming a minimally invasive diagnostic or surgical procedure on apatient 12 who is lying down on an operating table 14. The system caninclude a surgeon's console 16 for use by a surgeon 18 during theprocedure. One or more assistants 20 may also participate in theprocedure. The MIRS system 10 can further include a patient side cart 22(surgical robot) and an electronics cart 24. The patient side cart 22can manipulate at least one removably coupled tool assembly 26(hereinafter simply referred to as a “tool”) through a minimallyinvasive incision in the body of the patient 12 while the surgeon 18views the surgical site through the console 16. An image of the surgicalsite can be obtained by an imaging device 28, such as a stereoscopicendoscope, which can be manipulated by the patient side cart 22 so as toorient the imaging device 28. The electronics cart 24 can be used toprocess the images of the surgical site for subsequent display to thesurgeon 18 through the surgeon's console 16. The number of surgicaltools 26 used at one time will generally depend on the diagnostic orsurgical procedure and the space constraints within the operating roomamong other factors. If it is necessary to change one or more of thetools 26 being used during a procedure, an assistant 20 may remove thetool 26 from the patient side cart 22, and replace it with another tool26 from a tray 30 in the operating room.

MIRS system 10 in certain embodiments is a system for performing aminimally invasive diagnostic or surgical procedure on a patientincluding various components such as a surgeon's console 16, anelectronics cart 24, and a patient side cart 22. However, it will beappreciated by those of ordinary skill in the art that the system couldoperate equally well by having fewer or a greater number of componentsthan are illustrated in FIG. 1A. Thus, the depiction of the system 10 inFIG. 1A should be taken as being illustrative in nature, and notlimiting to the scope of the disclosure.

FIG. 1B diagrammatically illustrates a robotic surgery system 50 (suchas MIRS system 10 of FIG. 1A). As discussed above, a surgeon's console52 (such as surgeon's console 16 in FIG. 1A) can be used by a surgeon tocontrol a patient side cart (surgical robot) 54 (such as patient sidecart 22 in FIG. 1A) during a minimally invasive procedure. The patientside cart 54 can use an imaging device, such as a stereoscopicendoscope, to capture images of the procedure site and output thecaptured images to an electronics cart 56 (such as the electronics cart24 in FIG. 1A). As discussed above, the electronics cart 56 can processthe captured images in a variety of ways prior to any subsequentdisplay. For example, the electronics cart 56 can overlay the capturedimages with a virtual control interface prior to displaying the combinedimages to the surgeon via the surgeon's console 52. The patient sidecart 54 can output the captured images for processing outside theelectronics cart 56. For example, the patient side cart 54 can outputthe captured images to a processor 58, which can be used to process thecaptured images. The images can also be processed by a combination ofthe electronics cart 56 and the processor 58, which can be coupledtogether so as to process the captured images jointly, sequentially,and/or combinations thereof. One or more separate displays 60 can alsobe coupled with the processor 58 and/or the electronics cart 56 forlocal and/or remote display of images, such as images of the proceduresite, or other related images.

MIRS system 50 in certain embodiments is a system for performing aminimally invasive diagnostic or surgical procedure on a patientincluding various components such as a surgeon's console 52, anelectronics cart 56, and a patient side cart 54. However, it will beappreciated by those of ordinary skill in the art that the system couldoperate equally well by having fewer or a greater number of componentsthan are illustrated in FIG. 1B. Thus, the depiction of the system 50 inFIG. 1B should be taken as being illustrative in nature, and notlimiting to the scope of the disclosure.

FIG. 2 is a perspective view of the surgeon's console 16. The surgeon'sconsole 16 includes a left eye display 32 and a right eye display 34 forpresenting the surgeon 18 with a coordinated stereo view of the surgicalsite that enables depth perception. The console 16 further includes oneor more input control devices 36, which in turn cause the patient sidecart 22 (shown in FIG. 1A) to manipulate one or more tools. The inputcontrol devices 36 can provide the same degrees of freedom, or moredegrees of freedom, as their associated tools 26 (shown in FIG. 1A) soas to provide the surgeon with telepresence, or the perception that theinput control devices 36 are integral with the tools 26 so that thesurgeon has a strong sense of directly controlling the tools 26. To thisend, position, force, and tactile feedback sensors (not shown) may beemployed to transmit position, force, and tactile sensations from thetools 26 back to the surgeon's hands through the input control devices36.

The surgeon's console 16 is usually located in the same room as thepatient so that the surgeon may directly monitor the procedure, bephysically present if necessary, and speak to an assistant directlyrather than over the telephone or other communication medium. However,the surgeon can be located in a different room, a completely differentbuilding, or other remote location from the Patient allowing for remotesurgical procedures.

Surgeon's console 16 in certain embodiments is a device for presentingthe surgeon with information concerning the surgical site and receivinginput information from the surgeon, and includes various components suchas eyes displays and input control devices. However, it will beappreciated by those of ordinary skill in the art that the surgeon'sconsole could operate equally well by having fewer or a greater numberof components than are illustrated in FIG. 2. Thus, the depiction of thesurgeon's console 16 in FIG. 2 should be taken as being illustrative innature, and not limiting to the scope of the disclosure.

FIG. 3 is a perspective view of the electronics cart 24. The electronicscart 24 can be coupled with the imaging device 28 and can include aprocessor to process captured images for subsequent display, such as toa surgeon on the surgeon's console, or on another suitable displaylocated locally and/or remotely. For example, where a stereoscopicendoscope is used, the electronics cart 24 can process the capturedimages so as to present the surgeon with coordinated stereo images ofthe surgical site. Such coordination can include alignment between theopposing images and can include adjusting the stereo working distance ofthe stereoscopic endoscope. As another example, image processing caninclude the use of previously determined camera calibration parametersso as to compensate for imaging errors of the image capture device, suchas optical aberrations.

The electronics cart 24 in certain embodiments is a device forpresenting information concerning a surgery to a surgical team andincludes various components displays, processors, storage elements, etc.However, it will be appreciated by those of ordinary skill in the artthat the electronics cart could operate equally well by having fewer ora greater number of components than are illustrated in FIG. 3. Thus, thedepiction of the electronics cart 24 in FIG. 3 should be taken as beingillustrative in nature, and not limiting to the scope of the disclosure.

FIG. 4 shows a patient side cart 22 having a plurality of manipulatorarms, each supporting a surgical instrument or tool 26 at a distal endof the manipulator arm. The patient side cart 22 shown includes fourmanipulator arms 100 which can be used to support either a surgical tool26 or an imaging device 28, such as a stereoscopic endoscope used forthe capture of images of the site of the procedure. Manipulation isprovided by the robotic manipulator arms 100 having a number of roboticjoints, where each joint provides a manipulator degree of freedom. Theangle of each joint may be controlled by an actuator such as a motor ormotor assembly, and in some embodiments the angle of each joint may bemeasured using one or more sensors (e.g., encoders, or potentiometers,or the like) disposed on or proximate to each joint. The imaging device28 and the surgical tools 26 can be positioned and manipulated throughincisions in the patient so that a kinematic remote center is maintainedat the incision so as to minimize the size of the incision. Images ofthe surgical site can include images of the distal ends of the surgicalinstruments or tools 26 when they are positioned within thefield-of-view of the imaging device 28.

Regarding surgical tool 26, a variety of alternative robotic surgicaltools or instruments of different types and differing end effectors maybe used, with the instruments of at least some of the manipulators beingremoved and replaced during a surgical procedure. Several of these endeffectors, including DeBakey Forceps, microforceps, Potts scissors, andclip applier include first and second end effector elements which pivotrelative to each other so as to define a pair of end effector jaws.Other end effectors, including scalpel and electrocautery probe have asingle end effector element. For instruments having end effector jaws,the jaws will often be actuated by squeezing the grip members of handle.Single end effector instruments may also be actuated by gripping of thegrip members, for example, so as to energize an electrocautery probe.

The elongate shaft of instrument 26 allows the end effectors and thedistal end of the shaft to be inserted distally into a surgical worksitethrough a minimally invasive aperture, often through an abdominal wallor the like. The surgical worksite may be insufflated, and movement ofthe end effectors within the patient will often be affected, at least inpart, by pivoting of the instrument 26 about the location at which theshaft passes through the minimally invasive aperture. In other words,manipulators 100 will move the proximal housing of the instrumentoutside the patient so that shaft extends through a minimally invasiveaperture location so as to help provide a desired movement of endeffector. Hence, manipulators 100 will often undergo significantmovement outside the patient 12 during a surgical procedure.

The patient side cart 22 in certain embodiments is a device forproviding surgical tools for assisting in a surgical procedure on apatient, and may include various components such as manipulator arms 100and tools 26. However, it will be appreciated by those of ordinary skillin the art that the patient side cart could operate equally well byhaving fewer or a greater number of components than are illustrated inFIG. 4. Thus, the depiction of the patient side cart 22 in FIG. 4 shouldbe taken as being illustrative in nature, and not limiting to the scopeof the disclosure.

An exemplary manipulator arm in accordance with some embodiments of thepresent invention can be understood with reference to FIG. 5. Asdescribed above, a manipulator arm generally supports a distalinstrument or surgical tool and affects movements of the instrumentrelative to a base. As a number of different instruments havingdiffering end effectors may be sequentially mounted on each manipulatorduring a surgical procedure (typically with the help of a surgicalassistant), a distal instrument holder will preferably allow rapidremoval and replacement of the mounted instrument or tool. As can beunderstood with reference to FIG. 4, manipulators are proximally mountedto a base of the patient side cart. Typically, the manipulator armincludes a plurality of linkages and associated joints extending betweenthe base and the distal instrument holder. In one aspect, an exemplarymanipulator includes a plurality of joints having either redundant ornon-redundant degrees of freedom, but is lacking at least one degree offreedom necessary to fully prescribe the position (i.e., location andorientation) of the end effector.

In many embodiments, such as that shown in FIG. 5, an exemplarymanipulator arm includes a proximal revolute joint J1 that rotates abouta first joint axis so as to revolve the manipulator arm distal of thejoint about the joint axis. In some embodiments, revolute joint J1 ismounted directly to the base, while in other embodiments, joint J1 maybe mounted to one or more movable linkages or joints. The joints of themanipulator, in combination, may have redundant degrees of freedom suchthat the joints of the manipulator arm can be driven into a range ofdiffering configurations for a given end effector position. For example,the manipulator arm of FIG. 5 may be maneuvered into differingconfigurations while the distal instrument or tool 511 supported withinthe instrument holder 510 maintains a particular state, which mayinclude a given position or velocity of the end effector. In someembodiments, the joints of the manipulator are not operable toindependently control at least one of the six end effector degrees offreedom that fully define the position of the tool 511. For example, themanipulator may not be operable to cause the tool 511 to independentlyroll, pitch, yaw, and/or translate in one or more directions.

Describing the individual links of manipulator arm 500 of FIG. 5 alongwith the axes of rotation of the joints connecting the links asillustrated in FIG. 5, a first link 504 extends distally from a pivotaljoint J2 which pivots about its joint axis and is coupled to revolutejoint J1 which rotates about its joint axis. Many of the remainder ofthe joints can be identified by their associated rotational axes, asshown in FIG. 5. For example, a distal end of first link 504 is coupledto a proximal end of a second link 506 at a pivotal joint J3 that pivotsabout its pivotal axis, and a proximal end of a third link 508 iscoupled to the distal end of the second link 506 at a pivotal joint J4that pivots about its axis, as shown. The distal end of the third link508 is coupled to instrument holder 510 at pivotal joint J5. Typically,the pivotal axes of each of joints J2, J3, J4, and J5 are substantiallyparallel and the linkages appear “stacked” when positioned next to oneanother, so as to provide a reduced width of the manipulator arm andimprove patient clearance during maneuvering of the manipulatorassembly. In many embodiments, the instrument holder 510 also includesadditional joints, such as a prismatic joint J6 that facilitates axialmovement of the instrument 511 through the minimally invasive apertureand facilitates attachment of the instrument holder 510 to a cannulathrough which the instrument 511 is slidably inserted. In someembodiments, even when combining the degrees of freedom of theinstrument holder 510 with the rest of those of manipulator arm 500, theresulting degrees of freedom are still insufficient to provide at leastone of the six degrees of freedom necessary to fully define the positionof the tool 511.

The instrument 511 may include additional degrees of freedom distal ofinstrument holder 510. Actuation of the degrees of freedom of theinstrument will often be driven by motors of the manipulator, andalternative embodiments may separate the instrument from the supportingmanipulator structure at a quickly detachable instrumentholder/instrument interface so that one or more joints shown here asbeing on the instrument are instead on the interface, or vice versa. Insome embodiments, instrument 511 includes a rotational joint J7 (notshown) near or proximal of the insertion point of the tool tip or thepivot point PP, which generally is disposed at the site of a minimallyinvasive aperture. A distal wrist of the instrument allows pivotalmotion of an end effector of surgical tool 511 about instrument jointsaxes of one or more joints at the instrument wrist. An angle between endeffector jaw elements may be controlled independently of the endeffector location and orientation. Notwithstanding these additionalkinematic degrees of freedom provided by the surgical tool 511, whichmay be considered to be part of the manipulator degrees of freedom, insome embodiments, even when combining the kinematic degrees of freedomof the surgical tool 511 with those of manipulator arm 500 (including,e.g., those of instrument holder 510), the resulting kinematic degreesof freedom are still insufficient to fully control the position of thetip of tool 511.

The manipulator arm 500 in certain embodiments is a mechanical body forholding and controlling a tool, and may include a number of links andjoints. However, it will be appreciated by those of ordinary skill inthe art that the manipulator arm could operate equally well by havingfewer or a greater number of components than are illustrated in FIG. 5.Thus, the depiction of the manipulator arm 500 in FIG. 5 should be takenas being illustrative in nature, and not limiting to the scope of thedisclosure.

FIG. 6A shows a surgical tool 600 that includes a proximal chassis 602,an instrument shaft 604, and a distal end effector 606 having a jaw 608that can be articulated to grip a patient tissue. The proximal chassisincludes an input coupler that is configured to interface with and bedriven by an output coupler of the patient side cart 22 (FIG. 1A). Theinput coupler is drivingly coupled with an input link of a springassembly 610. The spring assembly 610 is mounted to a frame 612 of theproximal chassis 602 and includes an output link that is drivinglycoupled with a drive shaft that is disposed within the instrument shaft604. The drive shaft is drivingly coupled with the jaw 608.

In accordance with some embodiments and as shown in FIG. 6A, thesurgical tool 600 may not include any degrees of freedom for altering aposition of the end effector 606. In other embodiments, the surgicaltool 600 may include one or more joints for adding degrees of freedomfor altering the position of the end effector 606. For example, theinstrument shaft 604 may include joints for changing a pitch and/or yawof the end effector 606. Further, in some embodiments and as shown inFIG. 6A, the surgical tool 600 may include one or more degrees offreedom for actuating the end effector 606. For example, the springassembly 610 may be operable to actuate the jaw 608. Additionalcharacteristics of surgical tool 600, as well as other surgical tools,are described in commonly-owned U.S. application Ser. No. 13/297,158,filed Nov. 15, 2011, entitled “Method for Passively Decoupling TorqueApplied By a Remote Actuator Into an Independently Rotating Member,” thedisclosure of which is incorporated herein by reference in its entirety.

FIG. 6B illustrates a wristed endoscope 620 that may, in someembodiments, be used in robotic minimally invasive surgery. Theendoscope 620 includes an elongate shaft 622 and a flexible wrist 624located at the working end of the shaft 622. A housing 626 allows thesurgical instrument 620 to releasably couple to a manipulator located atthe opposite end of the shaft 624. An endoscopic camera lens isimplemented at the distal end of the flexible wrist 624. A lumen (notshown) runs along the length of the shaft 622 which connects the distalend of the flexible wrist 624 to the housing 626. In a “fiber scope”embodiment, imaging sensor(s) of the endoscope 620, such as chargecoupled devices (CCDs), may be mounted inside the housing 626 withconnected optical fibers running inside the lumen along the length ofthe shaft 622 and ending at substantially the distal end of the flexiblewrist 624. In an alternate “chip-on-a-stick” embodiment, the imagingsensor(s) of the endoscope 620 may be mounted at the distal end of theflexible wrist 624. The imaging sensor(s) may be two-dimensional orthree-dimensional.

In some embodiments, the flexible wrist 624 may have at least one degreeof freedom to allow the endoscope 620 to articulate and maneuver easilyaround internal body tissues, organs, etc. to reach a desireddestination (e.g., epicardial or myocardial tissue). The housing 626 mayhouse a drive mechanism for articulating the distal portion of theflexible wrist 624. The drive mechanism may be cable-drive, gear-drive,belt drive, or another type of drive mechanism suitable to drive theflexible wrist 624 along its degree(s) of freedom. For example, in oneembodiment, the flexible wrist 624 may have two translation degrees offreedom and the shaft 622 may be operable to rotate around an axis alongthe length of the shaft 622. In some medical procedures, the articulateendoscope 620 maneuvers and articulates around internal organs, tissues,etc. to acquire visual images of hard-to-see and/or hard-to-reachplaces. Additional characteristics of the endoscope 620, as well asother surgical tools, are described in commonly-owned U.S. applicationSer. No. 11/319,011, filed Dec. 27, 2005, entitled “Articulate andSwapable Endoscope for a Surgical Robot,” the disclosure of which isincorporated herein by reference in its entirety.

FIG. 6C is a perspective view of the distal end of an overtube withsuction ports. The overtube 630 defines an instrument lumen 632 whichextends through the overtube 630 to permit passage of an instrument. Theovertube 630 further comprises one or more suction passages 634 whichare coupled to a vacuum source. The overtube 630 may, in variousembodiments, be formed out of any of a variety of materials suitable forsurgical use and may be provided with any of variety of stiffnesses. Forexample, the overtube 630 may comprise a substantially rigid material,may comprise a flexible material, or may comprise a combination of oneor more substantially rigid portions and one or more flexible portionsto provide a bendable structure. The cross-sectional shape of theovertube 630 may also vary. In the illustrated embodiment, the overtube630 has a substantially circular cross-sectional shape and is made outof polyurethane. In other embodiments, other cross-sectional shapes maybe used, such as, e.g., oval, rectangular, triangular, etc., dependingon the application.

In the illustrated embodiment, the suction passages 634 comprise aplurality of vacuum lumens within the wall of the overtube 630, witheach vacuum lumen being coupled to the vacuum source (not shown). Thevacuum source may be operated to create a vacuum pressure in eachsuction passage 634, thereby creating a suction force onto a tissuesurface which the suction passages 634 are in contact with. As a resultof this suction force, the overtube 630 will be attached to the tissuesurface. If the vacuum pressure is discontinued, the tissue surface willbe released and the overtube 630 will no longer be attached to thetissue. Accordingly, by controllably providing a suction force via thesuction passages 634, the overtube 630 can be releasably attached topatient's tissue surface. A surgical instrument, such as an irrigationtool, cutting tool, etc., may then be inserted through the instrumentlumen 200 to treat tissue disposed within the instrument lumen 632.

In accordance with some embodiments, the overtube 630 may be made ofsubstantially rigid material and not include any degrees of freedom foraltering a position of the overtube 630. In other embodiments, theovertube 630 may include one or more joints for adding degrees offreedom for altering the position of the distal end of the overtube 630.For example, the overtube 630 may include joints for changing a pitchand/or yaw of the distal end of the overtube 630. Further, in someembodiments, the overtube 630 may include one or more degrees of freedomfor actuating functionality of the overtube 630. For example, a vacuumsource (not shown) may be operable to create or remove a vacuum pressurein one or more suction passages 634. Additional characteristics of theovertube 630, as well as other surgical tools, are described incommonly-owned U.S. application Ser. No. 11/618,374, filed Dec. 29,2006, entitled “Vacuum Stabilized Overtube for Endoscopic Surgery,” thedisclosure of which is incorporated herein by reference in its entirety.

FIG. 6D illustrates a non-wristed endoscope 640 that may, in someembodiments, be used in robotic minimally invasive surgery. Thenon-wristed endoscope 640 is similar to the wristed endoscope 620depicted in and discussed with reference to FIG. 6B, and thus similarlyincludes a housing 646 and a shaft 622. The difference is that thenon-wristed endoscope 640 does not include a flexible wrist. Thenon-wristed endoscope has a reduced number of degrees of freedomcompared to the wristed endoscope, and in this particular example,non-wristed endoscope 640 does not have a wrist pitch or wrist yaw.

The surgical tool 600, endoscope 620, and overtube 30 are various toolsthat include a variety of components. However, it will be appreciated bythose of ordinary skill in the art that these tools could operateequally well by having fewer or a greater number of components than areillustrated in FIGS. 6A to 6C. Further, it would will also beappreciated that other tools may also or alternatively be used, such asgripping devices, electrosurgical paddles, vacuums, irrigators,staplers, scissors, knifes, etc. Thus, the depiction of surgical toolsin FIGS. 6A to 6C should be taken as being illustrative in nature, andnot limiting to the scope of the disclosure.

FIG. 7A is a perspective view of a master control input device 700 thatmay be part of a surgeon's console 16 (FIG. 1A) in accordance with anembodiment. The master control 700 includes a gimbal or wrist 720 thatis operatively coupled to an articulated arm 740.

Master control input device 700 has a number of degrees of freedom andis operable to control a manipulator assembly (e.g., manipulator arm 500of FIG. 5). The degrees of freedom of input device 700 includeskinematic degrees of freedom defined by joints of input device 700, usedto control the kinematics of manipulator arm 500, and may also includeactuation degrees of freedom used to actuate a tool (e.g., instrument511) connected to manipulator arm 500. Input device 700, like a tool ofmanipulator arm 500, may also be considered to have an end effector (or,more generally, a control frame) associated therewith, which itself hasa number of kinematic degrees of freedom.

In some embodiments, input device 700 may have a sufficient number ofdegrees of freedom to fully control the position of an end effector. Forexample, the input device 700 may have six degrees of freedom that mayindependently control the three translation and three orientationdegrees of freedom of an end effector of the instrument 511. In somecases, even though the input device 700 has such a sufficient number ofdegrees of freedom, the manipulator assembly (e.g., manipulator arm 500)has a number of degrees of freedom that is insufficient to independentlycontrol the three translation and three orientation degrees of freedomof the end effector. For example, the manipulator arm 500 may have onlyfive degrees of freedom.

In some embodiments, the input device 700 may have additional degrees offreedom, which may be degrees of freedom operable to control theposition of the end effector (e.g., a redundant degree of freedom),and/or may be degrees of freedom operable to actuate the instrument 26(e.g., turning on or off suction or irrigation, actuating a clamp,engaging a staple with tissue, etc.). An input device having additionaldegrees of freedom is described in commonly-owned U.S. application Ser.No. 10/121,283, filed Apr. 11, 2002, entitled “Master Having RedundantDegrees of Freedom,” the disclosure of which is incorporated herein byreference in its entirety. Further, in at least one embodiment, theinstrument 511, either alone or in combination with a manipulator arm500, may have additional kinematic degrees of freedom that add to thedegrees of freedom of the manipulator arm 500. For example, theinstrument 511 may have joints for controlling the position of the endeffector. In some cases, even when combining the kinematic degrees offreedom of the manipulator arm 500 with the kinematic degrees of freedomof the instrument, the position of the end effector may not be fullycontrolled. This may be, e.g., due to the joints of the instrument 511merely adding kinematic degrees of freedom that are redundant to thosealready provided by the manipulator arm 500. In some embodiments, theinstrument 511 may have additional actuation degrees of freedom foractuating the instrument 511 (e.g., turning on or off suction orirrigation, actuating a clamp, engaging a staple with tissue, etc.).

To facilitate control of the instrument 511, the master control inputdevice 700 may include one or more actuators or motors and, in someembodiments, sensors for each of a plurality of joints of the mastercontrol input device 700. The motors and sensors of the input device 700may be operatively linked to the motors and sensors associated with themanipulator arms (e.g., arm 500 of FIG. 5) and the surgical instrumentsmounted thereon (e.g., instrument 511 of FIG. 5) via a control systemdisposed in, e.g., the surgeon's console 16, the electronics cart 24,and/or the patient cart 22, and/or any other element of MIRS system 10(FIG. 1). The control system may include one or more processors foreffecting control between the master control device input and responsiverobotic arm and surgical instrument output and for effecting controlbetween robotic arm and surgical instrument input and responsive mastercontrol output in the case of, e.g., force feedback.

FIG. 7B is a perspective view of a gimbal or wrist 720 according to anembodiment. According to this embodiment, gimbal or wrist 720 allowsrotation of an actuatable handle 722 about three axes, axis 1, axis 2,and axis 3. More specifically, the handle 722 is coupled to a firstelbow-shaped link 724 by a first pivotal joint 726. The first link 724is coupled to a second elbow-shaped link 728 by a second pivotal joint730. The second link 728 is pivotally coupled to a third elbow-shapedlink 732 by a third pivotal joint 734. The gimbal or wrist 720 may bemounted on an articulated arm 740 (as shown in FIG. 7A) at axis 4 suchthat the gimbal or wrist 720 can displace angularly about axis 4. By wayof such links and joints, the gimbal or wrist 720 may provide a numberof kinematic degrees of freedom for the control input device 700 and beoperable to control one or more of the end effector degrees of freedom.

In some embodiments, the handle 722 may include a pair of grip members723 for actuating a tool or end effector. For example, by opening orclosing the grip members 723, the jaw 608 of the end effector 606 (FIG.6) may similarly be opened or closed. In other embodiments, one or moreinput elements of the handle 722 and/or of other elements of thesurgeon's console 16 may be operable to actuate one or more degrees offreedom of the instrument 511 other than degrees of freedom forcontrolling the position of the instrument 26. For example, thesurgeon's console 16 may include a foot pedal coupled to the controlsystem for activating and deactivating a vacuum pressure.

In some embodiments, the joints of the gimbal or wrist 720 may beoperatively connected to actuators, e.g., electric motors, or the like,to provide for, e.g., force feedback, gravity compensation, and/or thelike. Furthermore, sensors such as encoders, potentiometers, and thelike, may be positioned on or proximate to each joint of the gimbal orwrist 720, so as to enable joint positions of the gimbal or wrist 720 tobe determined by the control system.

FIG. 7C is a perspective view of an articulated arm 740 according to anembodiment. According to this embodiment, the articulated arm 740 allowsrotation of a gimbal or wrist 720 (FIG. 7B) about three axes, axis A,axis B, and axis C. More specifically, the gimbal or wrist 720 may bemounted on the arm 740 at axis 4 as previously described with referenceto FIG. 7B. The gimbal or wrist 720 is coupled to a first link 742 whichis pivotally coupled to a second link 744 by a first pivotal joint 746.The second link 744 is pivotally coupled to a third link 748 by a secondpivotal joint 750. The third link 748 may be pivotally coupled to thesurgeon's console 16 (FIG. 1) by a third pivotal joint 752. By way ofsuch links and joints, the articulated arm 740 may provide a number ofkinematic degrees of freedom for the control input device 700 and beoperable to control one or more of the kinematic degrees of freedom of amanipulator assembly to thereby control the position of an instrument(e.g., instrument 511 of FIG. 5).

In some embodiments, the joints of the articulated arm 740 may beoperatively connected to actuators, e.g., electric motors, or the like,to provide for, e.g., force feedback, gravity compensation, and/or thelike. Furthermore, sensors such as encoders, potentiometers, and thelike, may be positioned on or proximate to each joint of the articulatedarm 740, so as to enable joint positions of the articulated arm 740 tobe determined by the control system.

Input device 700 in certain embodiments is a device for receiving inputsfrom a surgeon or other operator and includes various components such asa gimbal or wrist 720 and an articulated arm 740. However, it will beappreciated by those of ordinary skill in the art that the input devicecould operate equally well by having fewer or a greater number ofcomponents than are illustrated in FIGS. 7A to 7C. Thus, the depictionof the input device 700 in FIGS. 7A to 7C should be taken as beingillustrative in nature, and not limiting to the scope of the disclosure.

FIG. 8 is a block diagram showing a simplified system 800 forcontrolling a mechanical body having fewer degrees of freedom than thosemathematically modeled. The system 800 includes an information source810, a kinematic processor 820 having a kinematic model 822, one or moreactuators 830, and a mechanical body 840.

The information source 810 may be any suitable source of controlinformation 812 for controlling a position of the mechanical body. Inone embodiment, the information source 810 is an input device such asthe surgeons console 16 (FIG. 1), the input control devices 36 (FIG. 2),and/or the master control input device 700 (FIGS. 7A to 7C). In suchembodiments, the control information 812 may be a desired position of amechanical body having N degrees of freedom. In one embodiment, the Ndegrees of freedom are those necessary to fully define the position ofthe mechanical body, i.e., three independently controllable translationsand three independently controllable rotations. Accordingly, the outputfrom the information source 810 may include parameters for fullycontrolling the position of the mechanical body. In one embodiment, theN degrees of freedom are insufficient to fully define the position ofthe mechanical body. For example, the N degrees are freedom areinsufficient to independently control all three translations and threerotations of the mechanical body. Accordingly, the output from theinformation source 810 may include parameters insufficient to fullycontrol the position of the mechanical body. In yet another embodiment,the N degrees of freedom may include one or more degrees of freedom thatdo not define the position of the mechanical body (i.e., non-kinematicdegrees of freedom). For example, the N degrees of freedom may includeone or more degrees of freedom for actuating an instrument such asactuating a vacuum pressure. Accordingly, the output from theinformation source 810 may include parameters for controllingnon-kinematic characteristics of the mechanical body. These may not belimited to suction via a vacuum pressure, but may also or alternativelyinclude irrigating, energizing (e.g., cauterizing), cutting (using,e.g., a single blade or multiple blades like scissors), and grasping(using, e.g., pincers, fingers, or the like).

In some embodiments, when the information source 810 is an input device,the input device itself also has N degrees of freedom, such that, atleast in some cases, each degree of freedom of the input device maycorrespond to a degree of freedom of the mechanical body. For example, aroll of the input device may correspond to a desired roll of themechanical body, or a pitch of the input device may correspond to adesired pitch of the mechanical body. In other embodiments, the inputdevice may have a number of degrees of freedom greater than or fewerthan the mechanical body. For example, the input device may have one ormore redundant degrees of freedom, whereas the mechanical body may haveno redundant degrees of freedom. For another example, the mechanicalbody may have one or more redundant degrees of freedom, whereas theinput device may have no redundant degrees of freedom.

Although the input device provides an output indicating a desiredposition of a mechanical body having N degrees of freedom, themechanical body 840 controlled by the input device lacks at least one ofthe necessary degrees of freedom for fully defining the position of themechanical body. For example, the mechanical body 840 may lack a roll,yaw, and/or pitch degree of freedom, and/or may lack an up/down,left/right, and/or forward/backward translation degree of freedom. Inthe case where N represents the number of degrees of freedom forcontrolling a position of a mechanical body, and M represents the numberof degrees of freedom regarding position that the mechanical body lacks,then in one embodiment the mechanical body may have N−M degrees offreedom. For example, N may be equal to six, corresponding to all threetranslations and all three rotations of a mechanical body, whereas M mayrepresent one, corresponding to the roll of the mechanical body.However, in other embodiments, the mechanical body may have Nor evengreater than N degrees of freedom, such as where the mechanical bodyincludes redundant degrees of freedom, but still lacks at least onedegree of freedom necessary to fully define the position of themechanical body.

In other embodiments, the information source 810 is not an input devicelike a surgeon's console, but rather is a tool position measuringdevice. In this case, the kinematic block 820 is not a controller, butis a joint position estimator. For example, the tool position measuringdevice may include an optical fiber that extends the length of toolassembly 26 to the free end of the tool assembly 26, an electromagneticsensor arranged proximate the joints of the manipulator assembly, orother sensor or imaging device operable to measure a position of thejoints of the manipulator assembly. In many embodiments, the toolposition measuring device is small and light enough so as not tointerfere with motion of the tool. In an embodiment where the toolposition measuring device includes an optical fiber, the properties(e.g., the refractive index) of the optical fiber may be altered as aresult of changes in joint positions. Some examples of fiber opticsensors are described in U.S. Pat. App. Pub. No. US2007/0156019 A1(filed Jul. 20, 2006), entitled “Robotic Surgery System IncludingPosition Sensors Using Fiber Bragg Gratings” by Larkin et al., and U.S.patent application Ser. No. 12/164,829 (filed Jun. 30, 2008) entitled“Fiber optic shape sensor” by Giuseppe M. Prisco, both of which areincorporated herein by reference in their entirety for all purposes. Thetool position measuring device may then be operable to determine aposition of the tip of the tool (e.g., the free end of tool assembly 26)based on the altered properties of the optical fiber. The tool positionmeasuring device may further be operable to measure a position of thetool in a number of degrees of freedom. For example, the tool positionmeasuring device may measure one, two, or three translational positionsof the tool (e.g., x, y, z positions), and/or one, two or threeorientational positions of the tool (e.g., pitch, yaw, roll). In atleast one embodiment, the tool position measuring device may beinoperable to measure one or more degrees of freedom positions of thetool. For example, when the tool position measuring device includes anoptical fiber and deduces a tip position based on changes to theproperties of the optical fiber, the tool position measuring device mayhave difficulty determining a roll movement of the tool. In embodimentswhere the information source 810 is a tool position measuring device,control information 812 may be measurement information indicating ameasured position of the degrees of freedom of the tool tip.

In some embodiments, the tool position measuring device may be operableto measure the position of a number of degrees of freedom of the toolless than, equal to, or greater than a number of degrees of freedom ofthe manipulator assembly. In many embodiments, as already described forinformation sources being input devices, the mechanical body may have Ndegrees of freedom that are insufficient to fully define the position ofthe mechanical body, may include non-kinematic degrees of freedom, mayinclude redundant degrees of freedom, etc.

The output from the information source 810, regardless of whether theinformation source 810 is an input device or a tool position measuringdevice, is applied to a kinematic model 822 of an N degree of freedommechanical body in the kinematic processor 820. The kinematic processor820 may be provided in any suitable component of MIRS system 10 (FIG.1), such as the surgeon's console 16, electronics cart 24, and/orpatient side cart 22, tool assembly 26, manipulator assembly, and/or thecontrol system discussed with reference to FIG. 7A.

In embodiments where the information source 810 is an input device, thekinematic model 822 may be a model of a mechanical body having N degreesof freedom that correspond to the N degrees of freedom for which adesired control is output from the input device. For example, the inputdevice may output parameters for fully controlling the position of themechanical body, such as parameters for controlling three translationsand three rotations of the mechanical body. The kinematic model 822 maythen be a kinematic model of a mechanical body having three translationsand three rotations. In some embodiments, one or more degrees of freedomthat do not define the position of the mechanical body may be modeled orotherwise controlled separate from the kinematic model 822. Regardless,in most embodiments, the kinematic model 822 includes a mathematicalrepresentation of the one or more degrees of freedom lacking in themechanical body 840. For example, the mechanical body 840 may lack adegree of freedom for controlling a roll of the mechanical body 840.However, the kinematic model 822 may be of a mechanical body having adegree of freedom for controlling the roll of the mechanical body, andthe input device 810 may output information indicating a desired controlof the roll of the mechanical body.

In embodiments where the information source 810 is a tool positionmeasuring device, the kinematic model 822 may be a model of a mechanicalbody for which joint estimates (i.e., estimates of the position ofjoints corresponding to the degrees of freedom of the mechanical body840) are to be generated. For example, if a joint estimation techniqueis used in which the tool position measurement device can only provide Ndegrees of freedom measurement, then the kinematic model 822 may have atleast N degrees of freedom, the mechanical body may have (N−M) degreesof freedom, etc. However, in other embodiments, the mechanical body mayhave greater than N degrees of freedom, the manipulator assembly may belogically separated into multiple parts in which each part has no morethan N joints or kinematic degrees of freedom while the tool measurementtool device is used to measure the position and orientation at the endof each part.

As a result of applying the control information 812 from the informationsource 810 to the kinematic processor 820, one or more individualcontrol outputs 824 may be generated by the kinematic processor 820 andcommunicated to one or more actuators 830 operable to affect control ofone or more of the degrees of freedom of the mechanical body 840. Thenumber of individual control outputs 824 generated and communicated tothe actuators 830 is fewer than the total number of individual controloutputs 824 that may be generated by the kinematic processor 820. Thatis, the individual control outputs 824 communicated to the actuators 830do not include information for controlling the degree(s) of freedom thatthe mechanical body 840 lacks. For example, the kinematic model 822 maymodel a mechanical body having degrees of freedom for fully defining theposition of the mechanical body, and may calculate outputs forcontrolling all of those degrees of freedom. However, only a subset ofthose calculated outputs are used, as the mechanical body actuallycontrolled (i.e., mechanical body 840) does not have all of the degreesof freedom modeled by the kinematic model 822. Accordingly, theindividual control outputs 824 communicated to the actuators 830 areonly a subset of the possible instructions that may be generated usingthe kinematic model 822.

In embodiments where the information source 810 is an input device, theindividual control outputs 824 may include information indicating thedesired position of a degree of freedom of the mechanical body 840. Forexample, an individual control output 824 may indicate the desiredposition (e.g., angle) of a joint associated with one of the degrees offreedom of the mechanical body 840.

In embodiments where the information source 810 is a tool positionmeasuring device, the individual control outputs 824 may includeinformation indicating the actual position of a degree of freedom of themechanical body 840. For example, an individual control output 824 mayindicate the actual position (e.g., angle) of a joint associated withone of the degrees of freedom of the mechanical body 840.

The individual control outputs 824 are received by one or more actuators830 for controlling at least some of the degrees of freedom of amechanical body 840. For example, the actuators 830 may be electricmotors or the like operable to actuate joints of the mechanical body840, as previously discussed with reference to, e.g., FIG. 5. Inaccordance with one embodiment, each actuator is operable to control acorresponding degree of freedom of the mechanical body 840. However, inother embodiments, one actuator may be operable to control more than oneor fewer than one degree of freedom of the mechanical body 840.

The mechanical body 840 may be any suitable mechanical body having atleast one degree of freedom. For example, the mechanical body may be arobotic manipulator arm (e.g., manipulator arm 100 described withreference to FIG. 4 or manipulator arm 500 described with reference toFIG. 5) and/or surgical instrument (e.g., instrument 26 described withreference to FIG. 1A or instrument 511 described with reference to FIG.5). In some embodiments, the mechanical body may include the kinematicaspects of both a manipulator arm and surgical instrument.

As previously described, various embodiments incorporate informationsource 810 being an input device such that kinematic processor 820outputs desired positions of the joints of body 840, whereas otherembodiments incorporate information source 810 being a tool positionmeasuring device such that kinematic processor 820 outputs actualpositions of the joints of body 840. In yet other embodiments, a systemmay include both an input device for providing desired positions to akinematic model and controller (i.e., a type of kinematic processor)unique to the input device, and a tool position measuring device forproviding tool tip position information to a kinematic model and anestimator (i.e., a type of kinematic processor) unique to the toolposition measuring device. In such a case, the kinematic model used bythe tool tip position measuring device may be different than thekinematic model used by the input device. Further, the outputs of eachkinematic model, i.e., the desired position and actual position of thejoints of the manipulator assembly may be used together to calculate theactual amount of torque to be applied to each joint motor. Inembodiments where a tool position measuring device is not provided, thesystem may acquire estimated joint positions and combine those with thedesired positions output when using the input device to generate torqueamounts. Some of these further embodiments are described with referenceto FIG. 9.

System 800 in certain embodiments is a simplified system for controllinga mechanical body and includes various components such as an inputdevice 810, kinematic processor 820, actuator(s) 830, and mechanicalbody 840. However, it will be appreciated by those of ordinary skill inthe art that the system could operate equally well by having fewer or agreater number of components than are illustrated in FIG. 8. Thus, thedepiction of the system 800 in FIG. 8 should be taken as beingillustrative in nature, and not limiting to the scope of the disclosure.

FIG. 9 is a block diagram of an actuator 830 according an embodiment.The actuator 830 includes a joint controller 832 and a motor 834, wherethe joint controller 832 is operable to generator a torque command 836for controlling motor 834. The motor 834 may be coupled to one or morejoints of a manipulator assembly for controlling the degrees of freedomof the manipulator assembly. In this embodiment, the actuator 830 isoperable to control degree of freedom (X).

To generate a torque command 836, the joint controller 834 receives thedesired position of the degree of freedom (X) 824A. The desired positionmay be received from an input device such as a surgeon's console. Forexample, the desired position 824A may be generated by the kinematicprocessor 820 (FIG. 8) when the information source 810 is an inputdevice. Accordingly, the desired position 824A may be generated byapplying control information communicated from the input device to thekinematic model 822, and using only one of a subset of the individualcontrol outputs 824 as the desired position 824A.

To generate the torque command 836, the joint controller 834 alsoreceives the actual position of the degree of freedom (X) 824B. Theactual position may be generated using one or more of a number oftechniques. In one embodiment, the actual position may be determinedbased on encoders used for each joint motor. In other embodiments, theactual position may be calculated from the position (e.g., locationand/or orientation) of the end of the manipulator assembly (e.g., a tooltip) and applying inverse kinematics. A sensor device for sensing theposition of the end of the manipulator assembly may include an opticalfiber disposed along the length of the manipulator assembly with one endfixed at the tool tip such that changes in joint position cause changesto properties (e.g., refractive index) of the optical fiber. In anotherembodiment, the sensor device may include one or more electromagneticsensors attached to the end point of the manipulator assembly such thatany changes in the tip position could be measured from theelectromagnetic field generator. For example, with reference to FIG. 8,the information source 810 may be a sensing device operable to determinethe tool tip position, and by applying the tool tip position measurementto the kinematic model 822, one of a subset of the individual controloutput 824 may be used as the actual position 824B.

Upon receiving both the desired position and the actual position ofDOF(X), the joint controller 832 may determine the appropriate amount ofmotor torque that will cause the degree of freedom to move from itsactual (i.e., current) position to the desired position. The jointcontroller 832 then sends the torque command 836 indicating this amountof torque to the motor 834.

FIG. 10A is a block diagram showing a simplified system 1000 forcontrolling a manipulator assembly using an input device in accordancewith a first embodiment. System 1000 includes an input device 1010, acontroller 1020, and a manipulator assembly 1030. The input device 1010may be similar to the information source 810 discussed with reference toFIG. 8, controller 1020 may be similar to the kinematic processor 820discussed with reference to FIG. 8, and the manipulator assembly 1030may be similar to the actuators and mechanical body 840 discussed withreference to FIG. 8. In one embodiment, the manipulator assembly 1030includes a manipulator (e.g., manipulator 500) and/or a tool (e.g., tool511). Manipulator assembly 1030 may have one or more manipulatorkinematic degrees of freedom and, in some embodiments, may also oralternatively have one or more actuation degrees of freedom. Further,manipulator assembly 1030 may be operable to control the position of oneor more end effectors (or, more generally, control frames). For example,an end effector could be defined at a tip of a tool that is part ofmanipulator assembly 1030, part-way up a shaft of such a tool, etc. Theend effector also has a number of degrees of freedom which may be thesame as or different than the manipulator degrees of freedom.

In accordance with the embodiment depicted in FIG. 10A, the input device1010 and controller 1020 are operable to control a greater number ofkinematic manipulator degrees of freedom (i.e., degrees of freedom ofmanipulator assembly 1030) than the manipulator assembly 1030 actuallyhas. For example, the input device 1010 may have six kinematic degreesof freedom 1012 including three independently controllable rotationdegrees of freedom and three independently controllable translationdegrees of freedom. The input device 1010 outputs parameters or otherinformation corresponding to the position of the input device 1010 orotherwise indicating a desired position of an end effector associatedwith manipulator assembly 1030.

The output from the input device 1010 are received and processed by thecontroller 1020 to provide instructions for controlling the manipulatorassembly 1030 (e.g., instructions for controlling motors associated withjoints of manipulator assembly 1030). In this embodiment, the controller1020 includes a kinematic model 1022 of a manipulator assembly havingsix kinematic degrees of freedom for controlling three independentlycontrollable rotation and three independently controllable translationdegrees of freedom of the end effector.

A subset of the results from applying the output from the input device1010 to the kinematic model 1022 are then used to control themanipulator assembly 1030. For example, the subset of the results may becommunicated to one or more actuators associated with joints of themanipulator assembly 1030. The manipulator assembly 1030 in thisembodiment has four kinematic degrees of freedom. The manipulatorassembly 1030 is thus lacking two kinematic degrees of freedom. Forexample, the manipulator assembly 1030 may lack degrees of freedomcorresponding to yaw and pitch movements, or may lack degrees of freedomcorresponding to two translational movements, etc. Accordingly, thesubset includes instructions for controlling only those degrees offreedom that the manipulator assembly 1030 is configured to have. Inthis case, the instructions are indicative of a desired position of thejoints of manipulator assembly 1030, and may be combined withinformation indicating the actual position of the joints of themanipulator assembly 1030. The combination may be used to determine theappropriate torque to apply to the joint motors.

In other embodiments, the input device and kinematic model may not havesix kinematic degrees of freedom, and the end effector may not have fourkinematic degrees of freedom. Rather, the input device and kinematicmodel may be configured to control a greater number of degrees offreedom than the manipulator assembly is equipped with. For example, theinput device 1010 and controller 1020 may be configured to control fivekinematic degrees of freedom, whereas the manipulator assembly may haveanywhere from one to four kinematic degrees of freedom.

FIG. 10B is a block diagram showing a simplified system 1001 forcontrolling a manipulator assembly using an input device in accordancewith a second embodiment. System 1001 includes an input device 1010, acontroller 1020, and a manipulator assembly 1030, which may be similarto those discussed with reference to FIG. 10A.

In accordance with this embodiment, the input device 1010 and controller1020 are operable to control a greater number of kinematic degrees offreedom than the manipulator assembly 1030 has, similar to theembodiment discussed with reference to FIG. 10A. Further, the inputdevice 1010 is operable to actuate a non-kinematic degree of freedom ofthe manipulator assembly 1030. For example, the input device 1010 may beoperable to actuate a vacuum pressure associated with the manipulatorassembly 1030.

Accordingly, in this embodiment, the input device 1010 includes sixkinematic degrees of freedom 1012 as well as at least one actuationdegree of freedom 1014, where the actuation degree of freedom refers toa non-kinematic degree of freedom. In one embodiment, the actuationdegree of freedom may be actuated via an element on the input device1010 such as one or more grip members 723 (FIG. 7B). The input device1010, via the six kinematic degrees of freedom, may then outputparameters or other information corresponding to the position of theinput device 1010 or otherwise indicating a desired position of an endeffector associated with the manipulator assembly 1030. Further, theinput device 1010, via the one actuation degree of freedom, may outputparameters or other information for actuating a functionality of themanipulator assembly 1030 or a tool coupled to or part of manipulatorassembly 1030 (e.g., actuating a vacuum, one or more pincers/fingers,etc.).

The output from the input device 1010 are received and processed by thecontroller 1020 to provide instructions for controlling the manipulatorassembly 1030. In this embodiment, the controller 1020 includes akinematic model 1022 of a manipulator assembly having six kinematicdegrees of freedom, similar to that described with reference to FIG.10A. Further, the controller 1020 also includes an actuation controller1024 which may be operable to process the output from the input device1010 concerning the actuation degree of freedom 1014 and use this outputto actuate a function of the manipulator assembly 1030.

A subset of the results from applying the output from the input device1010 to the kinematic model 1022 are then used to control themanipulator assembly 1030, similar to that discussed with reference toFIG. 10A. Further, the results from applying the actuation output fromthe input device 1010 to the actuation controller 1024 may be used tocontrol actuation of the manipulator assembly 1030 (or actuation of atool coupled to or part of manipulator assembly 1030). As discussed withreference to FIG. 10A, in other embodiments, the input device andkinematic model may not have six kinematic degrees of freedom, and theend effector may not have four kinematic degrees of freedom. Further, insome embodiments, the input device 1010 may include a plurality ofactuation degrees of freedom, the actuation controller 1024 may beoperable to process the output for a plurality of actuation degrees offreedom from the input device 1010, and the manipulator assembly 1030may have a corresponding number of actuation degrees of freedom that maybe controlled by the input device 1010 separate from the control of themanipulator assembly 1030.

FIG. 10C is a block diagram showing a simplified system 1002 forcontrolling a manipulator assembly using an input device in accordancewith a third embodiment. System 1002 includes an input device 1010, acontroller 1020, and a manipulator assembly 1030, which may be similarto those discussed with reference to FIG. 10A.

In accordance with this embodiment, the manipulator assembly 1030 has anumber of kinematic degrees of freedom (e.g., eight), which includes atleast one null space degree of freedom (e.g., three null space degreesof freedom), but still lacks at least one of the degrees of freedomnecessary to fully define the position of an end effector associatedwith the manipulator assembly 1030. For example, although themanipulator assembly 1030 includes eight kinematic degrees of freedomincluding redundant degrees of freedom, it still lacks at least oneindependently controllable rotation or translation degree of freedom.

It should be recognized that when null space degrees of freedom of amanipulator assembly are described herein, it is assumed that thecontrol frame is located on the manipulator assembly (e.g., the controlframe is located at a tool tip). However, embodiments are not limited tosuch cases, and thus where null space degrees of freedom of amanipulator assembly are described, embodiments alternatively includenull space degrees of freedom of a Jacobian, where the Jacobian isassociated with the manipulator assembly and a control frame having anarbitrarily defined location, such as at a tissue of a patient, at afixed distance from the tool tip, etc.

The input device 1010 includes six kinematic degrees of freedom 1012similar to those discussed with reference to FIG. 10A. The controller1020 is then operable to process the output from the input device 1010by applying the output to a kinematic model 1026 including ninekinematic degrees of freedom, which includes six kinematic degrees offreedom such as those discussed with reference to FIG. 10A as well asthree null space degrees of freedom. The controller 1020 may thusgenerate outputs on the assumption that the manipulator assembly 1030includes nine kinematic degrees of freedom that include three null spacedegrees of freedom. However, since the manipulator assembly 1030 onlyincludes eight kinematic degrees of freedom, the controller 1020 doesnot provide an output corresponding to the missing kinematic degree offreedom of the manipulator assembly 1030. Rather, the controller 1020outputs instructions for controlling the eight kinematic degrees offreedom of the manipulator assembly 1030.

FIG. 10D is a block diagram showing a simplified system 1003 forcontrolling a manipulator assembly using an input device in accordancewith a fourth embodiment. System 1003 includes an input device 1010, acontroller 1020, and a manipulator assembly 1030, which may be similarto those discussed with reference to FIG. 10A.

In accordance with this embodiment, the input device 1010 includes oneor more null space degrees of freedom, whereas the manipulator assembly1030 does not include any null space degrees of freedom and includesfewer kinematic degrees of freedom than the input device 1010. In otherembodiments, the manipulator assembly 1030 may also include the same orgreater number of null space degrees of freedom as the input device1010.

Accordingly, in this embodiment, the input device 1010 includes a numberof degrees of freedom 1016 including seven kinematic degrees of freedomwhich include one null space degree of freedom. The output from theinput device 1010 may then be processed and eventually fed into thecontroller 1020, where the controller 1020 includes a kinematic model ofa manipulator assembly having five degrees of freedom. The controller1020 may then apply the output from the input device 1010 to thekinematic model 1028, and then use a subset of the results to controlthe manipulator assembly 1030, similar to that discussed with referenceto FIG. 10A.

In this embodiment, the manipulator assembly 1030 includes fourkinematic degrees of freedom 1032. However, in other embodiments, themanipulator assembly 1030 may have fewer than four kinematic degrees offreedom. Further, while the input device 1010 is described as having onenull space degree of freedom, the input device 1010 may have more thanone null space degree of freedom. For example, the input device 1010 mayhave two, three, or four null space degrees of freedom.

Systems 1000, 1001, 1002, and 1003 in certain embodiments are simplifiedsystems for controlling an end effector using an input device andinclude various components such as an input device 1010, controller1020, and manipulator assembly 1030. However, it will be appreciated bythose of ordinary skill in the art that the systems could operateequally well by having fewer or a greater number of components than areillustrated in FIGS. 10A to 10D. For example, in some embodiments, inputdevices may have both null space degrees of freedom and actuationdegrees of freedom, in addition to one or more kinematic degrees offreedom, controllers may have both null space degrees of freedom and anactuation controller, and manipulator assemblies may have both nullspace degrees of freedom and actuation degrees of freedom. Thus, thedepiction of the systems in FIGS. 10A to 10D should be taken as beingillustrative in nature, and not limiting to the scope of the disclosure.

FIG. 11A is a block diagram showing a simplified system 1100 forcontrolling a manipulator assembly using a tool position measuringdevice in accordance with a first embodiment. System 1100 includes atool position measuring device 1110, a joint estimator 1120, and amanipulator assembly 1130.

The elements of system 1100 are similar to the similarly labeledelements of system 1000, except that a tool position measuring device1110 is provided instead of an input device 1010, and a joint estimator1120 is provided instead of a controller. Accordingly, the descriptionwith reference to system 1000 is equally applicable to system 1100,except in the case of system 1100 instead of the controller receivingand applying desired position information, the controller receives andapplies position measurement information. And, instead of generatingdesired positions, the joint estimator 1120 generates actual positions(e.g., joint angles) of the manipulator assembly joints. In someembodiments, the actual positions may be combined with informationindicating the desired position of the joints of the manipulatorassembly 1130. The combination may be used to determine the appropriatetorque to apply to the joint motors.

Further, the tool position measuring device 1100 may only measure fivedegrees of freedom or less. In this particular embodiment, the toolposition measuring device 1100 is illustrated as measuring five degreesof freedom, however, it may similarly measure four degrees of freedom,three degrees of freedom, or less than three degrees of freedom. And,the joint estimator 1120 uses only a 5-kinematic DOF model 1122 whichgenerates only five joint positions, four of which are used (as theycorrespond to actual joints of the manipulator assembly 1130) and one ofwhich is discarded (as there is no corresponding joint in themanipulator assembly 1130). In some embodiments, a 6-kinematic DOF model1122 could be used in which two joint estimates would then be discarded,or fewer than five kinematic DOF could be used.

FIG. 11B is a block diagram showing a simplified system 1102 forcontrolling a manipulator assembly using a tool position measuringdevice in accordance with a second embodiment. System 1102 includes atool position measuring device 1110, a joint estimator 1120, and amanipulator assembly 1130.

The elements of system 1102 are similar to the similarly labeledelements of system 1002, except that a tool position measuring device1110 is provided instead of an input device 1010, and a joint estimator1120 is provided instead of a controller. Accordingly, the descriptionwith reference to system 1002 is equally applicable to system 1102,except in the case of system 1102 instead of the controller receivingand applying desired position information the joint estimator receivesand applies position measurement information. And, instead of generatingdesired positions, the joint estimator 1120 generates actual positions(e.g., joint angles) of the manipulator assembly joints. In someembodiments, the actual positions may be combined with informationindicating the desired position of the joints of the manipulatorassembly 1130. The combination may be used to determine the appropriatetorque to apply to the joint motors.

Further, in this particular embodiment, the manipulator assembly isshown as having eight kinematic degrees of freedom with multiple pitchand yaw joints. For some embodiments of manipulator assemblies, themanipulator assembly may be logically separated into multiple parts. Inthe embodiments where optical fiber is used to measure the position andorientation at the end of each part, each part has five or fewer jointsor degrees of freedom as the tool position measuring device 1112(a) or1112(b) measures five degrees of freedom. The parts may each have thesame or different number of degrees of freedom. For example, in thisembodiment, the manipulator assembly is logically separated into twoparts each having four degrees of freedom.

In at least one embodiment, the number of joints in each part may bemaximized prior to incorporating joints into other parts. For example,for a manipulator assembly having seven degrees of freedom, one part mayhave five degrees of freedom (i.e., the maximum number) and another partmay have the remaining degrees of freedom, i.e., two degrees of freedom.For another example, for a manipulator assembly having twelve degrees offreedom, two parts may each have five degrees of freedom, while a thirdpart has only two degrees of freedom. In some cases, joint angles forone or more parts may be computed using geometries rather than inversekinematics, where it is more computationally efficient to do so. Forexample, it may be more computationally efficient to use geometries overinverse kinematics when calculating joint estimations for a part havinga number of degrees of freedom equal to or less than two.

The tool position measuring device 1110 may be operable to measure aposition for each logical part of the manipulator assembly. For example,the tool position measuring device 1110 may measure a tip position ofthe first part and a tip position of the second part. The tip positionof each part may be measured in the same or different number of degreesof freedom, where the measured degrees of freedom may be less than, thesame, or greater than the degrees of freedom of the corresponding part.In this particular example, the tip position of the first part ismeasured in five degrees of freedom 1112(a), and the tip position of thesecond part is similarly measured in five degrees of freedom 1112(b). Itshould be recognized that these need not be the same, and in someembodiments may be any number less than five.

The joint estimator 1120 may then include a kinematic model for eachlogical part of the manipulator assembly. In this embodiment, jointestimator 1120 includes a first kinematic model 1126(a) and a secondkinematic model 1126(b). The first kinematic model 1126(a) is akinematic model of the first part of the manipulator assembly 1130, andthe second kinematic model 1126(b) is a kinematic model of the secondpart of the manipulator assembly 1130. Each kinematic model receives theoutput from the tip measurement corresponding to its respectivemanipulator assembly part. For example, the output of the firstmeasurement 1112(a) is applied to the first kinematic model 1126(a), andthe output of the second measurement 1112(b) is applied to the secondkinematic model 1126(b). The first kinematic model 1126(a) then outputsthe actual position of the first part of the manipulator assembly 1130,whereas the second kinematic model 1126(b) outputs the actual positionof the second part of the manipulator assembly 1130.

It should be apparent that the degrees of freedom of at least one of thekinematic models 1126(a) and 1126(b) may be greater than the actualnumber of degrees of freedom of the corresponding manipulator assemblypart. In this particular embodiment, although not necessary, bothkinematic models have a greater number of degrees of freedom than theircorresponding manipulator assembly part. That is, the first kinematicmodel 1126(a) has five degrees of freedom whereas the first part ofmanipulator assembly 1130 only has four degrees of freedom, and likewisefor the second kinematic model 1126(b) and second part of manipulatorassembly 1130. This extra degree of freedom is used to generate anoutput but, similar to other extra kinematic model degrees of freedomdescribed herein, is not subsequently used to determine the actualposition of the manipulator assembly parts.

It should be recognized that while the systems described with referenceto FIGS. 11A and 11B are considered similar for purposes of description,the embodiments described with reference to FIGS. 10A to 10D aredirected to systems for generating desired positions of manipulatorassembly joints, whereas the embodiments described herein with referenceto FIGS. 11A and 11B are directed to systems for generating actualpositions of manipulator assembly joints. In some embodiments and asalready described, these systems may be combined into one system. Forexample, the input device and controller of FIGS. 10A to 10D may be usedto generate a desired position, the tool position measuring device andjoint estimator of FIGS. 11A and 11B may be used to generate an actualposition, and these generated positions may be used in combination asdescribed with reference to FIG. 9.

FIG. 12A is a manipulator assembly 1200 according to an embodiment.Manipulator assembly 1200 may be similar to manipulator assembly 1130described with reference to FIG. 11B. Manipulator assembly 1200 includesa number of links 1202 and a number of joints 1204. The manipulatorassembly 1200 is logically separated into a first part 1206 and a secondpart 1208. The first part 1206 extends from a point on the manipulatorassembly, farthest from the free end of the manipulator assembly, thatis defined as the base 1210. The first part 1206 extends from the base1210 to a point within the manipulator assembly identified as the tipposition of the first part 1212. The second part 1208 then extends fromthe tip position of the first part 1212 to a point defined as the tipposition of the second part 1214 which, in this example, is located atthe free end of the manipulator assembly 1200.

A number of joints are located within the first part 1206 and a numberof different joints are located within the second part 1208. The numbermay not be the same, but in this example each of the first and secondpart include four joints. In other examples, where the total number ofjoints is eight, the first and second parts may respectively includefive and three, or three and five joints. Various other combinations formanipulator assemblies having more than five joints may also beimplemented. Further, the manipulator assembly may be logicallyseparated into more than two parts. For example, when the total numberof joints is eight, a first part may have five joints, a second part mayhave two joints, and a third part may have one joint. In mostembodiments, when a joint estimation technique is used in which the toolposition measurement device can only provide N degrees of freedommeasurement, then each part includes no more than N joints or degrees offreedom. Further, when a manipulator assembly includes more than Nkinematic degrees of freedom, those degrees of freedom are separatedinto multiple logical parts. In one embodiment, where a fiber opticapproach is used in which estimation of the roll orientation is notavailable, then each part may include no more than five joints orkinematic degrees of freedom.

Turning briefly to FIG. 12B, FIG. 12B depicts a block diagram 1250illustrating the calculation of joint positions of multiple manipulatorassembly parts according to an embodiment. A joint estimator 1252 inthis example includes a first kinematic model 1254 and a secondkinematic model 1256. The first kinematic model 1254 is a kinematicmodel of the first segment or part of the manipulator assembly. Forexample, this may be a kinematic model of first part 1206. The secondkinematic model 1256 is a kinematic model of the second segment or partof the manipulator assembly. For example, this may be a kinematic modelof second part 1208.

Each of the kinematic models includes a phantom degree of freedom; thatis, a degree of freedom that does not exist in the corresponding part ofthe manipulator assembly. For example, in the embodiment depicted inFIG. 12A, each of the kinematic models may include a phantom roll degreeof freedom as the estimation of the roll orientation may not beavailable when using the fiber optic approach. In other embodiments,however, one or more of the kinematic models may include more than onephantom degree of freedom. In some embodiments, only one kinematic modelincludes a phantom degree of freedom.

The tip position of the first segment is input into the first kinematicmodel 1254. For example, the tip position 1212 may be input into firstkinematic model 1254. The output from first kinematic model 1254 is thejoint positions of first segment 1206 (as well as one set of outputs,corresponding to the phantom degree of freedom, that can be ignored).The difference between the tip position of the first segment (e.g., tipposition 1212) and the tip position of the second segment (e.g., tipposition 1214) is input into the second kinematic model 1256. The outputfrom second kinematic model 1256 is the joint positions of secondsegment 1208 (as well as one set of outputs, corresponding to thephantom degree of freedom, that can be ignored).

FIG. 13 is a flowchart showing a process 1300 for controllingmanipulator arms, tools, and/or end effectors using an input deviceaccording to a first embodiment. The manipulator arms, tools, and/or endeffectors may be any of those described herein, such as manipulator arms100 (FIG. 4), manipulator arms 500 (FIG. 5), tools 26 (FIG. 1A),surgical tool 600 (FIG. 6A), endoscope 620 (FIG. 6B), overtube 630 (FIG.6C), actuators 830 and/or mechanical body 840 (FIG. 8), etc. The inputdevice may be any of the input devices described herein, such as inputdevice 36 (FIG. 2), input device 700 (FIGS. 7A to 7C), input device 810(FIG. 8), etc. Further, the process 1000 may be performed by any of thecontrollers described herein, such as the control system discussed withreference to FIG. 7A, kinematic processor 820 (FIG. 8), and/or any othersuitable controller provided in any suitable component of MIRS system 10(FIG. 1), such as the surgeon's console 16, electronics cart 24, and/orpatient side cart 22.

In one particular embodiment, kinematic degrees of freedom of amanipulator assembly may be controlled by driving one or more joints viathe controller using motors of the system, the joints being drivenaccording to coordinated joint movements calculated by a processor ofthe controller. Mathematically, the controller may perform at least someof the calculations of the joint commands using vectors and/or matrices,some of which may have elements corresponding to configurations orvelocities of the joints. The range of alternative joint configurationsavailable to the processor may be conceptualized as a joint space. Thejoint space may, for example, have as many dimensions as the manipulatorassembly has degrees of freedom, and in some exemplary embodiments, thejoint space may have more dimensions than the manipulator assembly hasdegrees of freedom as the manipulator assembly may lack at least onedegree of freedom necessary to fully define the position of an endeffector associated with the manipulator assembly. Further, a particularconfiguration of the manipulator assembly may represent a particularpoint in the joint space, with each coordinate corresponding to a jointstate of an associated joint of the manipulator assembly where anassociated joint of the manipulator exists.

In an exemplary embodiment, the system includes a controller in which acommanded position and velocity of a feature in the work-space, denotedhere as its Cartesian space, are inputs. The feature may be any featureon the manipulator assembly or off the manipulator assembly which can beused as a control frame to be articulated using control inputs. Anexample of a feature on the manipulator assembly, used in many examplesdescribed herein, would be the tool-tip. Another example of a feature onthe manipulator assembly would be a physical feature which is not on thetool-tip, but is a part of the manipulator assembly, such as a pin or apainted pattern. An example of a feature off the manipulator assemblywould be a reference point in empty space which is exactly a certaindistance and angle away from the tool-tip. Another example of a featureoff the manipulator assembly would be a target tissue whose positionrelative to the manipulator assembly can be established. In all thesecases, the end effector is associated with an imaginary control framewhich is to be articulated using control inputs. However, in thefollowing, the “end effector” and the “tool tip” are used synonymously.Although generally, there is no closed form relationship which maps adesired Cartesian space end effector position to an equivalentjoint-space position, there is generally a closed form relationshipbetween the Cartesian space end effector and joint-space velocities. Thekinematic Jacobian is the matrix of partial derivatives of Cartesianspace position elements of the end effector with respect to joint spaceposition elements. In this way, the kinematic Jacobian captures thekinematic relationship between the end effector and the joints of themanipulator assembly. In other words, the kinematic Jacobian capturesthe effect of joint motion on the end effector. The kinematic Jacobian(J) can be used to map joint-space velocities (dq/dt) to Cartesian spaceend effector velocities (dx/dt) using the relationship below:dx/dt=Jdq/dtThus, even when there is no closed-form mapping between input and outputpositions, mappings of the velocities can iteratively be used, such asin a Jacobian-based controller, to implement a movement of themanipulator from a commanded user input. However, a variety ofimplementations can be used. Although many embodiments include aJacobian-based controller, some implementations may use a variety ofcontrollers that may be configured to access the Jacobian to provide anyof the features described herein.

One such implementation is described in simplified terms below. Thecommanded joint position is used to calculate the Jacobian (J). Eachtime step (Δt) calculates a Cartesian space velocity (dx/dt) to performthe desired move (dx_(des)/dt) and to correct for built up deviation(Δx) from the desired Cartesian space position. This Cartesian spacevelocity is then converted into a joint-space velocity (dq/dt) using thepseudo-inverse of the Jacobian (J^(#)). The resulting joint-spacecommanded velocity is then integrated to produce joint-space commandedposition (q). These relationships are listed below:dx/dt=dx _(des) /dt+kΔx  (1)dq/dt=J ^(#) dx/dt  (2)q _(i) =q _(i-1) +dq/dtΔt  (3)

The pseudo-inverse of the Jacobian (J#) directly maps the desired tooltip motion (and, in some cases, a remote center of pivotal tool motion)into the joint velocity space. If the manipulator assembly being usedhas more useful joint axes than tool tip (i.e., end effector) degrees offreedom (up to six), (and when a remote center of tool motion is in use,the manipulator assembly should have an additional 3 joint axes for the3 degrees of freedom associated with location of the remote center),then the manipulator assembly is said to be redundant. A Jacobianassociated with a redundant manipulator assembly includes a “null-space”having a dimension of at least one. In this context, the “null-space” ofthe Jacobian (N(J)) is the space of joint velocities whichinstantaneously achieves no tool tip motion (and when a remote center isused, no movement of the pivotal point location), and “null-motion” isthe path of joint positions which also produces no instantaneousmovement of the tool tip and/or location of the remote center.Incorporating or injecting the calculated null-space velocities into thecontrol system of the manipulator assembly to achieve the desiredreconfiguration of the manipulator assembly (including anyreconfigurations described herein) changes above equation (2) to thefollowing:dq/dt=dq _(perp) /dt+dq _(null) /dt  (4)dq _(perp) /dt=J ^(#) dx/dt  (5)dq _(null) /dt=(I−J ^(#) J)z=V _(n) V _(n) ^(T) z=V _(n)α  (6)

The joint velocity according to Equation (4) has two components: thefirst being the null-perpendicular-space component, the “purest” jointvelocity (shortest vector length) which produces the desired tool tipmotion (and when the remote center is used, the desired remote centermotion); and the second being the null-space component. Equations (2)and (5) show that without a null-space component, the same equation isachieved. Equation (6) starts with a traditional form for the null-spacecomponent on the left, and on the far right side, shows the form used inan exemplary system, wherein (V_(n)) is the set of orthonormal basisvectors for the null-space, and (α) are the coefficients for blendingthose basis vectors. In some embodiments, α is determined by knobs thatare used to shape the motion within the null-space as desired.

As previously mentioned, fully controlling the position of a rigid bodyrequires six independently controllable degrees of freedom, three fortranslations and three for orientations. This lends itself nicely to aJacobian based control algorithm, such as that discussed above, in whicha 6×N Jacobian matrix is used. However, some rigid bodies lack at leastone of these degrees of freedom. For example, a rigid endoscope tipwithout an articulating wrist is missing two degrees of freedom at thewrist, specifically wrist pitch and yaw. So it only has four degrees offreedom at the tip. This creates a problem for the 6×N Jacobianapproach, because the problem is now overconstrained. Using the 6×NJacobian based controller, when the endoscope tip is commanded to eitherpan or tilt, since it has a non-wristed tip, it can only do one thing,and this is to do a combination of both. This results in a sluggishunresponsive feel which is undesirable. Accordingly, not only is itdesirable to obviate this unresponsive feel, it is also desirable to usea 6×N Jacobian approach because then the same computation engine and/orkinematic model that is used for other arms and instruments can also beused for the camera arm as well.

Accordingly, in some embodiments, Equations (2) and (3) discussed abovemay be modified. First, Equation (2) may be modified by using a numberof phantom degrees of freedom corresponding to the missing degrees offreedom of the controlled mechanical body. This would extend the lengthof the (dq/dt) vector to equal to the sum of the numbers of existingdegrees of freedom plus phantom degrees of freedom. For example, phantomdegrees of freedom may be included in Equation (2), where those phantomDOF may be operable to control the wrist pitch and yaw for controllingthe above-described endoscope. By using phantom degrees of freedom, theJacobian based controller is faked into doing the pseudo-inversecalculation for a full six degree of freedom endoscope tip, i.e., awristed endoscope. The output of this is a set of joint velocities forcontrolling a six degree of freedom endoscope, even though the actualendoscope being controlled only has four independently controllabledegrees of freedom.

Second, in accordance with Equation (3), the joint positions arecalculated by integrating joint velocities. However, Equation (3) may bemodified such that the velocities of the phantom degrees of freedom,e.g., the nonexistent endoscope wrist joints, are not integrated andtherefore remain at a fixed position. In some embodiments, the fixed ordesired position may be set to any suitable value independent of a poseof the manipulator. For example, the fixed position may be set to 0degrees, 15 degrees, 30 degrees, 45 degrees, a value in the range of 0degrees to 45 degrees, a value less than 0 degrees or a value greaterthan 45 degrees.

By modifying Equations (2) and (3) for the control algorithm discussedabove, in the embodiment concerning the endoscope the endoscope tip mayconsequently follow an instructed command well without unnecessarysluggishness. The unwristed endoscope has no wrist joints to actuate,and therefore the wrist may stay straight. Further, if there is a forcereflection from the slave back to the masters, then the masters may becommanded to follow the straight tip of the endoscope, advantageouslyresulting in intuitive behavior.

In some embodiments, a pitch and yaw of the endoscope may beindependently controlled, but the endoscope may not be able toindependently roll. In response to an instruction to pan, a tip of theendoscope may be panned using only the pitch and yaw degrees of freedom.By using pitch and yaw, instead of translations and roll, the endoscopemay be controlled to pan while substantially maintaining a location atan aperture of the patient. For example, the endoscope may be controlledto pan without increasing the size or placing pressure on an aperture ofthe patient through which the endoscope is disposed. This may be done,for example, by pivoting the endoscope about a pivot point at theaperture (i.e., access site).

It should be recognized that advantages are not limited to increasingthe responsiveness of controlled tools and increasing the flexibility ofthe system by using the same controller to operate tools havingdifferent degrees of freedom. Rather, in some embodiments, advantagesmay be realized where tools may be actuated without requiring anyadditional degrees of freedom on an input device.

For example, in some embodiments, there may only be four inputs at themanipulator assembly, where three are typically used to control movementsuch as roll, pitch, and yaw, and the fourth is typically used tocontrol a single actuation of an instrument (e.g. suction activation).However, it may be desired to control two actuations of an instrument(e.g., suction activation and irrigation activation) using the samenumber of inputs at the manipulator assembly. By using a kinematic modelthat calculates tool instructions using all three kinematic degrees offreedom, i.e., roll, pitch, and yaw, but then discarding one of theoutputs, such as roll, motion of the tool may be controlled using onlytwo inputs, i.e., pitch and yaw. The other two inputs may then be usedto control two actuations of the instrument, such as suction activationand irrigation activation. Accordingly, although the instrument has onlyfour degrees of freedom in total, including both movement and actuationdegrees of freedom, as a result of using phantom degrees of freedom inthe controller, the instrument appears to have five degrees of freedom.In some embodiments, phantom degrees of freedom may be used on axiallysymmetrical instruments, which may advantageously further increase theillusion to the operator of the system (e.g., a surgeon) that they arecontrolling a degree of freedom which actually may not exist in theinstrument.

Returning now to FIG. 13, in operation 1310, the controller calculatesthe forward kinematics from the manipulator's joint positions. As aresult of this calculation, the controller determines the commandedCartesian space velocity (dx_(des)/dt), the commanded Cartesian spaceposition (x_(des)), the actual Cartesian space position (x), and theerror between the latter two (dx=x_(des)−x). To calculate the forwardkinematics, the controller may use the previously commanded jointposition (e.g., the variable (q) calculated in an immediately precedingtime step). In operation 1320, the controller calculates the desiredmove (dx/dt) using Equation (1). To calculate the desired move, thecontroller may use the output from step 1310 as well as the commandedend effector position (x_(des)). In operation 1330, the controllercalculates the Jacobian (J), where calculating the Jacobian (J) uses thepreviously commanded joint position (q). In operation 1340, thecontroller calculates the pseudo-inverse of the Jacobian (J^(#)).

In operation 1350, the controller calculates the joint-space velocity(dq/dt) using the pseudo-inverse of the Jacobian (J^(#)) calculated inoperation 1340 and using the desired move (dx/dt) calculated inoperation 1320. The pseudo-inverse of the Jacobian in this operationincludes phantom degrees of freedom as previously discussed. That is,the pseudo-inverse of the Jacobian includes mathematical representationsof degrees of freedom of a mechanical body even though those degrees offreedom may not actually exist on the mechanical body being controlledby the controller. Then, in operation 1360, the controller calculatesthe joint-space commanded position (q) using Equation (3) and thejoint-space velocity (dq/dt) calculated in operation 1350. However, aspreviously discussed, the velocities of the phantom degrees of freedomare not integrated in this operation and thus remain at (or may be setto) a fixed position.

Those skilled in the art would recognize that the operations discussedwith reference to FIG. 13 may be executed frequently so as to providereal-time control of an instrument responsive to a user input. Forexample, the operations may be performed a plurality of times eachsecond, in some embodiments about 1,000 times per second, 1,300 timesper second, 1,500 times per second, in a range from 1,000 times persecond to 1,500 times per second, less than 1,000 times per second ormore than 1,500 times per second.

It should be appreciated that the specific operations illustrated inFIG. 13 provide a particular method of controlling manipulator arms,tools, and/or end effectors, according to certain embodiments of thepresent invention. Other sequences of operations may also be performedaccording to alternative embodiments. For example, alternativeembodiments of the present invention may perform the operations outlinedabove in a different order. Moreover, the individual operationsillustrated in FIG. 13 may include multiple sub-operations that may beperformed in various sequences as appropriate to the individualoperation. Furthermore, additional operations may be added or existingoperations removed depending on the particular applications. One ofordinary skill in the art would recognize and appreciate manyvariations, modifications, and alternatives.

FIG. 14 is a flowchart showing a process 1400 for controllingmanipulator arms, tools, and/or end effectors using an input deviceaccording to a second embodiment. The manipulator arms, tools, etc.,input device and controller for executing the process 1400, may besimilar to those described above with reference to FIG. 13, and thusfurther details are omitted.

In contrast to the process 1300 described with reference to FIG. 13, theprocess 1400 may be operable to calculate and control a null space of aJacobian associated with a manipulator assembly. For example, themanipulator assembly may have one or more redundant degrees of freedom,but may still use one or more phantom joints where the manipulatorassembly, even with its redundant degrees of freedom, lacks one or moreof those degrees of freedom necessary to fully define the position of anend effector or tool.

Operations 1410 to 1440 are similar to operations 1310 to 1340 describedwith reference to FIG. 13, and thus further description is omitted. Inoperation 1450, the controller calculates the joint velocity componentwithin the null-perpendicular-space (dq_(perp)/dt) using thepseudo-inverse of the Jacobian (J^(#)) calculated in operation 1440 andthe Cartesian space velocity (dx/dt) calculated in operation 1420. Inoperation 1460, the controller calculates the joint velocity componentwithin the null-space (dq_(null)/dt) using the Jacobian calculated inoperation 1430 and the pseudo-inverse of the Jacobian (J^(#)) calculatedin operation 1440 and shown in Equation (6), or in some embodiments,using the singular value decomposition of the Jacobian (SVD(J)) or, insome embodiments, using the null-space basis vectors (V_(n)) andblending coefficients (α) as shown in Equation (6), or using any otherequivalent technique. In at least one embodiment, the output ofoperation 1450 may be used to calculate the joint velocity componentwithin the null-space (dq_(null)/dt) operation 1460. Similar to thatdiscussed with reference to operation 1350 and Equation (2), the jointvelocity component within the null-perpendicular-space (dq_(perp)/dt)and the joint velocity component within the null-space (dq_(null)/dt)may be calculated using phantom degrees of freedom in the Jacobian(e.g., in the pseudo-inverse of the Jacobian). Accordingly, each ofthese components of the joint-space velocity may be calculated using aJacobian that mathematically represents degrees of freedom that may notactually exist in the controlled manipulator.

In operation 1470, the controller calculates the commanded joint-spacevelocity (dq/dt) by summing the joint velocity component within thenull-perpendicular-space—(dq_(perp)/dt) and the joint velocity componentwithin the null-space (dq_(null)/dt) components calculated in operations1450 and 1460 and as shown in Equation (4). Since each of the componentsof the commanded joint-space velocity (dq/dt) were calculated to includeone or more phantom degrees of freedom, the resulting joint-spacevelocity (dq/dt) also includes one or more phantom degrees of freedom.Operation 1480 is then similar to operation 1360 described withreference to FIG. 13, and thus further description is omitted.

It should be appreciated that the specific operations illustrated inFIG. 14 provide a particular method of controlling manipulator arms,tools, and/or end effectors, according to certain embodiments of thepresent invention. Other sequences of operations may also be performedaccording to alternative embodiments. For example, alternativeembodiments of the present invention may perform the operations outlinedabove in a different order. Moreover, the individual operationsillustrated in FIG. 14 may include multiple sub-operations that may beperformed in various sequences as appropriate to the individualoperation. Furthermore, additional operations may be added or existingoperations removed depending on the particular applications. One ofordinary skill in the art would recognize and appreciate manyvariations, modifications, and alternatives.

One skilled in the art would also recognize that while the processes ofFIGS. 13 and 14 were described with reference to embodiments where inputinformation is a desired position and comes from an input device (likethe embodiments described with reference to FIGS. 10A to 10D), theprocesses may equally be applicable to embodiments where inputinformation is an actual position and comes from a tool positionmeasuring device (like the embodiments described with reference to FIGS.11A and 11B). In such embodiments, instead of using a commanded endeffector position (x_(des)) (e.g., as an input to operations 1320 and1420), an actual end effector position would be used. And instead ofgenerating a commanded joint position (q) (e.g., as an output ofoperation 1360 and 1480), an actual joint position would be generated.

The operations described in this application may be implemented assoftware code to be executed by one or more processors using anysuitable computer language such as, for example, Java, C, C++ or Perlusing, for example, conventional, sequential, or object-orientedtechniques. The software code may be stored as a series of instructions,or commands on a computer-readable medium, such as a random accessmemory (RAM), a read-only memory (ROM), a magnetic medium such as ahard-drive or a floppy disk, flash memory, or an optical medium such asa CD-ROM. Any such computer-readable medium may also reside on or withina single computational apparatus, and may be present on or withindifferent computational apparatuses within a system or network.

The present invention can be implemented in the form of control logic insoftware, firmware, or hardware or a combination of these. The controllogic may be stored in an information storage medium as a plurality ofinstructions adapted to direct an information processing device toperform a set of steps disclosed in embodiments of the presentinvention. Based on the disclosure and teachings provided herein, aperson of ordinary skill in the art will appreciate other ways and/ormethods to implement the present invention.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing embodiments (especially in the context of thefollowing claims) are to be construed to cover both the singular and theplural, unless otherwise indicated herein or clearly contradicted bycontext. The terms “comprising,” “having,” “including,” and “containing”are to be construed as open-ended terms (i.e., meaning “including, butnot limited to,”) unless otherwise noted. The term “connected” is to beconstrued as partly or wholly contained within, attached to, or joinedtogether, even if there is something intervening. Recitation of rangesof values herein are merely intended to serve as a shorthand method ofreferring individually to each separate value falling within the range,unless otherwise indicated herein, and each separate value isincorporated into the specification as if it were individually recitedherein. All methods described herein can be performed in any suitableorder unless otherwise indicated herein or otherwise clearlycontradicted by context. The use of any and all examples, or exemplarylanguage (e.g., “such as”) provided herein, is intended merely to betterilluminate embodiments and does not pose a limitation on the scopeunless otherwise claimed. No language in the specification should beconstrued as indicating any non-claimed element as essential to thepractice of at least one embodiment.

Preferred embodiments are described herein, including the best modeknown to the inventors. Variations of those preferred embodiments maybecome apparent to those of ordinary skill in the art upon reading theforegoing description. The inventors expect skilled artisans to employsuch variations as appropriate, and the inventors intend for embodimentsto be constructed otherwise than as specifically described herein.Accordingly, suitable embodiments include all modifications andequivalents of the subject matter recited in the claims appended heretoas permitted by applicable law. Moreover, any combination of theabove-described elements in all possible variations thereof iscontemplated as being incorporated into some suitable embodiment unlessotherwise indicated herein or otherwise clearly contradicted by context.The scope of the invention should, therefore, be determined not withreference to the above description, but instead should be determinedwith reference to the pending claims along with their full scope orequivalents.

What is claimed is:
 1. A method of operating a surgical system, themethod comprising: measuring a position of a surgical instrument, thesurgical instrument being included in a mechanical assembly having aplurality of joints and first degrees of freedom, the position of thesurgical instrument being measured for each of second degrees of freedomof the surgical instrument to determine position measurementscorresponding to the second degrees of freedom; estimating, with atleast one computer, a position of each joint of the plurality of joints,wherein estimating the position of each joint of the plurality of jointsincludes applying the position measurements to a kinematic model of themechanical assembly, the kinematic model having third degrees offreedom, the third degrees of freedom including a corresponding degreeof freedom for each of the first degrees of freedom and furtherincluding at least one specific degree of freedom that is missing fromthe first degrees of freedom, the application of the positionmeasurements to the kinematic model including a forward-kinematicsimulation of each of the third degrees of freedom for at least one timestep, and the estimating of the position of each joint of the pluralityof joints including a discarding of values for the at least one specificdegree of freedom from a result of the forward-kinematic simulation; andcontrolling the mechanical assembly based on the estimated positions ofthe plurality of joints.
 2. The method of claim 1, wherein applying theposition measurements to the kinematic model results in a number ofposition estimates greater than the number of the plurality of joints,and out of all of the position estimates, only a subset of the positionestimates are used to control the mechanical assembly.
 3. The method ofclaim 2, further comprising communicating each position estimate in thesubset of position estimates to a joint controller associated with acorresponding one of the plurality of joints.
 4. The method of claim 1,wherein controlling the mechanical assembly includes combining theestimated position of a first joint of the plurality of joints with adesired position of the first joint.
 5. The method of claim 1, whereinthe second degrees of freedom correspond to three translationalmovements and three rotational movements excluding a roll movement. 6.The method of claim 5, wherein the third degrees of freedom include theroll movement.
 7. The method of claim 1, wherein the number of thirddegrees of freedom is less than or equal to the number of degrees offreedom necessary to fully define the position of the surgicalinstrument.
 8. The method of claim 1, wherein the at least one specificdegree of freedom includes at least one translational or rotationaldegree of freedom of the surgical instrument.
 9. The method of claim 1,wherein the at least one specific degree of freedom includes at leasttwo rotational degrees of freedom of the surgical instrument.
 10. Themethod of claim 1, wherein: the kinematic model includes a firstkinematic model of a first part of the mechanical assembly and a secondkinematic model of a second part of the mechanical assembly; measuring aposition of the surgical instrument includes measuring a position of thefirst part of the mechanical assembly and measuring a position of thesecond part of the mechanical assembly; and estimating a position ofeach of the joints includes applying the position measurements of thefirst part to the first kinematic model and applying the positionmeasurements of the second part to the second kinematic model.
 11. Themethod of claim 1, wherein the at least one specific degree of freedomincludes a pitch rotation and a yaw rotation of the surgical instrument.12. The method of claim 1, wherein the at least one specific degree offreedom includes a roll rotation about a symmetrical axis of thesurgical instrument.
 13. A surgical system for performing minimallyinvasive surgery through an aperture of a patient, the systemcomprising: a manipulator assembly including a surgical tool, themanipulator assembly having a plurality of joints and first degrees offreedom, the manipulator assembly being operable to position thesurgical tool at the aperture of the patient; a tool position measuringdevice operable to measure a position of the surgical tool for each ofsecond degrees of freedom of the surgical tool and to determine positionmeasurements corresponding to the second degrees of freedom; and acontroller operable to perform operations including: estimating aposition of each joint of the plurality of joints, wherein estimatingthe position of each joint of the plurality of joints includes applyingthe position measurements to a kinematic model of the manipulatorassembly, the kinematic model having third degrees of freedom, the thirddegrees of freedom including a corresponding degree of freedom for eachof the first degrees of freedom and further including at least onespecific degree of freedom that is missing from the first degrees offreedom, the application of the position measurements to the kinematicmodel including a forward-kinematic simulation of each of the thirddegrees of freedom for at least one time step, and the estimating of theposition of each joint of the plurality of joints including a discardingof values for the at least one specific degree of freedom from a resultof the forward-kinematic simulation; and controlling the manipulatorassembly based on the estimated positions of the plurality of joints.14. The surgical system of claim 13, wherein the tool position measuringdevice includes an optical fiber extending to a tip of the surgicaltool.
 15. The surgical system of claim 13, wherein applying the positionmeasurements to the kinematic model results in a number of positionestimates greater than the number of the plurality of joints, and thecontroller is operable to control the manipulator assembly by using, outof all of the position estimates, only a subset of the positionestimates.
 16. The surgical system of claim 13, further comprising asurgeon's console including an input device for receiving an input fromthe surgeon, wherein the controller is operable to control themanipulator assembly based on the estimated positions of the joints andthe input received from the input device.
 17. The surgical system ofclaim 16, wherein applying the position measurements to the kinematicmodel results in a number of position estimates greater than the numberof the plurality of joints, and the controller is operable to controlthe manipulator assembly by combining a subset of the position estimateswith the input received from the input device.
 18. The surgical systemof claim 13, wherein the number of third degrees of freedom is less thanor equal to the number of degrees of freedom necessary to fully definethe position of the surgical tool.
 19. The surgical system of claim 13,wherein the number of first degrees of freedom is greater than or equalto the number of degrees of freedom necessary to fully define theposition of the surgical tool, and estimating the position of each oneof the plurality of joints includes applying some of the positionmeasurements to a first kinematic model of a first part of themanipulator assembly and other of the position measurements to a secondkinematic model of a second part of the manipulator assembly.
 20. Amethod for controlling the movement of a mechanical body, the methodcomprising: receiving control information for controlling the positionof a mechanical body, the mechanical body having first degrees offreedom; generating, with at least one computer, a plurality ofindividual control outputs by applying the received control informationto a kinematic model for the mechanical body, the kinematic model havingsecond degrees of freedom, the second degrees of freedom including acorresponding degree of freedom for each of the first degrees of freedomand further including at least one specific degree of freedom that ismissing from the first degrees of freedom, and the application of thereceived control information to the kinematic model including aforward-kinematic simulation of each of the second degrees of freedomfor at least one time step; selecting a subset of the individual controloutputs to control the first degrees of freedom, the subset beingselected by a discarding of values for the at least one specific degreeof freedom from a result of the forward-kinematic simulation; andtransmitting the subset of individual control outputs for use incontrolling the first degrees of freedom of the mechanical body.
 21. Themethod of claim 20, wherein the control information is received from aninput device and includes information indicating a desired position ofthe mechanical body.
 22. The method of claim 21, wherein the subset ofindividual control outputs indicates a desired position of the firstdegrees of freedom.
 23. The method of claim 22, further comprisinggenerating, for each of the first degrees of freedom, a torqueinstruction indicating an amount of torque to apply to a joint motoroperable to control the degree of freedom by combining an individualcontrol output that corresponds to the degree of freedom with jointposition information indicating a position of a joint corresponding tothe degree of freedom.
 24. The method of claim 20, wherein the controlinformation is received from a tool position measuring device andincludes measurements of each of third degrees of freedom positions of atool coupled to the mechanical body.
 25. The method of claim 24, whereinthe subset of individual control outputs indicates an actual position ofthe first degrees of freedom.
 26. The method of claim 25, furthercomprising generating, for each of the first degrees of freedom, atorque instruction indicating an amount of torque to apply to a jointmotor operable to control the degree of freedom by combining anindividual control output that corresponds to the degree of freedom withdesired position information indicating a desired position of a jointcorresponding to the degree of freedom.